model-serving

Unified Model Serving Framework 🍱

In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

Standardized Serverless ML Inference Platform on Kubernetes

Machine Learning automation and tracking

Hopsworks - Data-Intensive AI platform with a Feature Store

💧 Versatile Data Pipeline (VDP) is an open-source tool to seamlessly integrate AI for unstructured data into the modern data stack

A scalable inference server for models optimized with OpenVINO™

Model Deployment at Scale on Kubernetes 🦄️

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

A high-performance serving framework for ML models, offers dynamic batching and multi-stage pipeline to fully exploit your compute machine

Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services working together via the Lightning Apps framework.

FastAPI Skeleton App to serve machine learning models production-ready.

Code samples for the Lightbend tutorial on writing microservices with Akka Streams, Kafka Streams, and Kafka

A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.

Common library for serving TensorFlow, XGBoost and scikit-learn models in production.

A scalable, high-performance serving system for federated learning models

BentoML Example Projects 🎨

flink-jpmml is a fresh-made library for dynamic real time machine learning predictions built on top of PMML standard models and Apache Flink streaming engine

Serving PyTorch models with TorchServe 🔥

ClearML - Model-Serving Orchestration and Repository Solution