The deployment framework aims to provide a simple, lightweight, fast integrated, pipelined deployment framework that ensures reliability, high concurrency and scalability of services.

Overview

Savior

save your time.

只在Ubuntu18.04下完成全部测试,其他平台暂时未测试。

目前项目还处于早期开发阶段,如有任何问题,欢迎添加微信nsnovio,备注部署,进群交流。

背景

savior是一个能够进行快速集成算法模块并支持高性能部署的轻量开发框架。能够帮助将团队进行快速想法验证(PoC),避免重复的去github上找模型然后复现模型;能够帮助团队将功能进行流程拆解,很方便的提高分布式执行效率;能够有效减少代码冗余,减少不必要负担。

workflow的已经做好的轮子很多,例如perfectpolyaxondagster等。 之所以开发一个新的,主要原因是那些框架都太heavy了,对于大部分用户来说没法直接白嫖。

这个项目的核心目的就是能够减少大家的重复性开发,能够把绝大部分能够直接白嫖的东西放在框架里面,然后大家专注于自己的业务属性上,提升大家的工作效率。

特性

  1. 弹性伸缩:用户可以根据目前的请求量,自定义配置机器数。方便项目上量,并且保证服务器资源吃满(支持K8S)。
  2. 流程DAG:用户通过自定义自己的流程,框架支持DAG,保证流程的最高的并行度。
  3. 容灾能力强:集群中所有节点都是相同作用,不会因为部分节点挂掉而服务崩溃。
  4. 可扩展性强:框架主要是实现了一种设计模式,开发者只需要按照当前设计模式,扩展性无上限。
  5. 部署便捷:部署到上线不会超过5分钟(不考虑网速)。

依赖的第三方组件

  • rabbitmq:用于celery进行分布式的任务分发

  • triton:用于gpu端的模型服务的集中部署

  • milvus:用于特征向量搜索,存储【推荐有搜索需求的用户自行配置】

    如果觉得milvus太大,用户可以根据自己的自身情况直接使用faiss或者nmslib。并且自己实现对应helper。

框架中已集成的算法

更多开源模型欢迎在issue中补充,也十分欢迎您的PR。

人脸相关

OCR相关

  • DB 文本检测
  • CRNN 文本识别
  • 版式分析
  • 文本图像方向矫正
  • 文本方向检测
  • 常见扇形环形转换为矩形(针对于segmentation base的检测方案)

图像搜索

通用

  • NRIQA

官方已适配模型下载地址(不定时更新):

根据自己的需要下载模型,不用全部下载。

简单使用教程

  1. 克隆项目git clone https://github.com/novioleo/Savior.git到本地。或者下载release下面的source包。
  2. 启动rabbitmq,推荐使用docker启动:docker run --restart=always -d --hostname celery-broker --name celery-broker -p5672:5672 -p15672:15672 -e RABBITMQ_DEFAULT_USER=guest -e RABBITMQ_DEFAULT_PASS=guest rabbitmq:3-management
  3. 启动triton,推荐使用docker(需要安装nvidia-docker)启动:docker run --gpus=all --name=triton-server -p8000:8000 -p8001:8001 -v/path/to/your/model/repo/path:/models nvcr.io/nvidia/tritonserver:20.12-py3 tritonserver --model-repository=/models,其中/path/to/your/model/repo/path是网盘中triton文件夹下载的所在文件夹。
  4. 修改项目配置,进入Savior文件夹中,进入Deployment包中,复制server_config.py.template并重命名为server_config.py,修改里面triton、rabbitmq的配置。
  5. 配置python与安装依赖,通过控制台进入Savior文件夹中,创建环境:conda create -n SaviorEnv python=3.8,激活环境source activate SaviorEnv,安装依赖:python -m pip install nvidia-pyindex==1.0.6 && python -m pip install -r requirements.txt
  6. 启动ConsumerWorker,通过控制台进入Savior文件夹中,启动worker:celery -A Deployment.ConsumerWorker worker --loglevel=INFO,如果一切配置正确会显示已经成功加载Task。
  7. 启动DispatchServer,通过控制台进入Savior文件夹中,启动server:python Deployment/DispathServer.py,启动成功会看到端口信息等。
  8. 测试接口服务,推荐使用apifox进行接口调用测试,可以通过post请求测试ocr_interface/general_ocr接口,传入参数image_url,发送请求(第一次运行需要等待,模型需要预热,五次之后基本上时间会稳定),会得到一个OSS的路径,如果OSS使用的是Dummy(默认),则找到/tmp/DummyOSS-temp-directory/{bucket_name}/{path}对应的文件。

生产级使用教程点我

接口结果预览

OCR相关

自然场景下OCR

如何在自有项目下开发?

移步至:DevelopTutorial

感谢

感谢各位开源项目大佬的无私奉献。

Owner
Tao Luo
Algorithmer.
Tao Luo
PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR)

Ilya Kostrikov 3k Dec 31, 2022
Space-event-trace - Tracing service for spaceteam events

space-event-trace Tracing service for TU Wien Spaceteam events. This service is

TU Wien Space Team 2 Jan 04, 2022
Fuzzing JavaScript Engines with Aspect-preserving Mutation

DIE Repository for "Fuzzing JavaScript Engines with Aspect-preserving Mutation" (in S&P'20). You can check the paper for technical details. Environmen

gts3.org (<a href=[email protected])"> 190 Dec 11, 2022
내가 보려고 정리한 <프로그래밍 기초 Ⅰ> / organized for me

Programming-Basics 프로그래밍 기초 Ⅰ 아카이브 Do it! 점프 투 파이썬 주차 강의주제 비고 1주차 Syllabus 2주차 자료형 - 숫자형 3주차 자료형 - 문자열형 4주차 입력과 출력 5주차 제어문 - 조건문 if 6주차 제어문 - 반복문 whil

KIMMINSEO 1 Mar 07, 2022
[NeurIPS 2021] "G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminators"

G-PATE This is the official code base for our NeurIPS 2021 paper: "G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of T

AI Secure 14 Oct 12, 2022
Optimizaciones incrementales al problema N-Body con el fin de evaluar y comparar las prestaciones de los traductores de Python en el ámbito de HPC.

Python HPC Optimizaciones incrementales de N-Body (all-pairs) con el fin de evaluar y comparar las prestaciones de los traductores de Python en el ámb

Andrés Milla 12 Aug 04, 2022
AquaTimer - Programmable Timer for Aquariums based on ATtiny414/814/1614

AquaTimer - Programmable Timer for Aquariums based on ATtiny414/814/1614 AquaTimer is a programmable timer for 12V devices such as lighting, solenoid

Stefan Wagner 4 Jun 13, 2022
Assessing syntactic abilities of BERT

BERT-Syntax Assesing the syntactic abilities of BERT. What Evaluate Google's BERT-Base and BERT-Large models on the syntactic agreement datasets from

Yoav Goldberg 147 Aug 02, 2022
Tensorflow Implementation for "Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition"

Tensorflow Implementation for "Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition" Pre-trained Deep Convo

Ankush Malaker 5 Nov 11, 2022
Laplacian Score-regularized Concrete Autoencoders

Laplacian Score-regularized Concrete Autoencoders Requirements: torch = 1.9 scikit-learn = 0.24 omegaconf = 2.0.6 scipy = 1.6.0 matplotlib How to

JS 6 Dec 07, 2022
The Official Implementation of Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose [NIPS 2021].

Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose Release Notes The offical PyTorch implementation of Neural View Sy

Angtian Wang 20 Oct 09, 2022
EDCNN: Edge enhancement-based Densely Connected Network with Compound Loss for Low-Dose CT Denoising

EDCNN: Edge enhancement-based Densely Connected Network with Compound Loss for Low-Dose CT Denoising By Tengfei Liang, Yi Jin, Yidong Li, Tao Wang. Th

workingcoder 115 Jan 05, 2023
SwinTrack: A Simple and Strong Baseline for Transformer Tracking

SwinTrack This is the official repo for SwinTrack. A Simple and Strong Baseline Prerequisites Environment conda (recommended) conda create -y -n SwinT

LitingLin 196 Jan 04, 2023
Video Frame Interpolation without Temporal Priors (a general method for blurry video interpolation)

Video Frame Interpolation without Temporal Priors (NeurIPS2020) [Paper] [video] How to run Prerequisites NVIDIA GPU + CUDA 9.0 + CuDNN 7.6.5 Pytorch 1

YoujianZhang 31 Sep 04, 2022
A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

This repository holds NVIDIA-maintained utilities to streamline mixed precision and distributed training in Pytorch. Some of the code here will be included in upstream Pytorch eventually. The intenti

NVIDIA Corporation 6.9k Jan 03, 2023
An open framework for Federated Learning.

Welcome to Intel® Open Federated Learning Federated learning is a distributed machine learning approach that enables organizations to collaborate on m

Intel Corporation 397 Dec 27, 2022
SEJE Pytorch implementation

SEJE is a prototype for the paper Learning Text-Image Joint Embedding for Efficient Cross-Modal Retrieval with Deep Feature Engineering. Contents Inst

0 Oct 21, 2021
Python module providing a framework to trace individual edges in an image using Gaussian process regression.

Edge Tracing using Gaussian Process Regression Repository storing python module which implements a framework to trace individual edges in an image usi

Jamie Burke 7 Dec 27, 2022
Deep Multimodal Neural Architecture Search

MMNas: Deep Multimodal Neural Architecture Search This repository corresponds to the PyTorch implementation of the MMnas for visual question answering

Vision and Language Group@ MIL 23 Dec 21, 2022
FastFace: Lightweight Face Detection Framework

Light Face Detection using PyTorch Lightning

Ömer BORHAN 75 Dec 05, 2022