serving

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for accelerating ML workloads.

A flexible, high-performance serving system for machine learning models

The open big data serving engine. https://vespa.ai

In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

Serve, optimize and scale PyTorch models in production

TensorFlow template application for deep learning

DELTA is a deep learning based natural language and speech processing platform.

⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for Cloud and Edge. Including Vision, Text, Audio and Video 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization and multi-platform multi-framework support.

Generic and easy-to-use serving service for machine learning models

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

A unified end-to-end machine intelligence platform

A scalable inference server for models optimized with OpenVINO™

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

ML pipeline orchestration and model deployments on Kubernetes, made really easy.

[Machine Learning Engineer Basic Guide] 부스트캠프 AI Tech - Product Serving 자료

MLOps Platform

Blockchain Search with GraphQL APIs

MLModelCI is a complete MLOps platform for managing, converting, profiling, and deploying MLaaS (Machine Learning-as-a-Service), bridging the gap between current ML training and serving systems.

bring keras-models to production with tensorflow-serving and nodejs + docker 🍕