#

llm-serving

Here are 12 public repositories matching this topic...

ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for accelerating ML workloads.

Updated Jul 13, 2023
Python

bentoml / OpenLLM

Operating LLMs in production

machine-learning ai deployment falcon developer-tools easy-to-use dolly fine-tuning t5-model llm model-inference lmops llmops llm-serving chatglm stablelm

Updated Jul 13, 2023
Python

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

inference pytorch transformer gpt model-serving mlops llm llmops llm-serving

Updated Jul 13, 2023
Python

skypilot-org / skypilot

SkyPilot: Run LLMs, AI, and Batch jobs on any cloud. Get maximum savings, highest GPU availability, and managed execution—all with a simple interface.

Updated Jul 13, 2023
Python

mosec

mosecorg / mosec

A high-performance serving framework for ML models, offers dynamic batching and multi-stage pipeline to fully exploit your compute machine

python rust machine-learning deep-learning tensorflow gpu pytorch hacktoberfest model-serving nerual-network machine-learning-platform jax mlops llm llm-serving

Updated Jul 8, 2023
Python

ray-project / aviary

Ray Aviary - evaluate multiple LLMs easily

distributed-systems transformers ray serving large-language-models llm llms llmops llm-serving llm-inference

Updated Jul 10, 2023
Python

ray-project / ray-educational-materials

This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.

deep-learning ray distributed-machine-learning ray-tune ray-train ray-distributed llm generative-ai ray-serve ray-data llm-serving llm-inference

Updated Jul 13, 2023
Jupyter Notebook

ray-project / llms-in-prod-workshop-2023

Deploy and Scale LLM-based applications

ray anyscale llm llms llm-serving llm-inference

Updated Jun 15, 2023
Jupyter Notebook

ray-project / anyscale-berkeley-ai-hackathon

Ray and Anyscale for UC Berkeley AI Hackathon!

hackathon berkeley-ai ray-distributed anyscale llm llm-serving llm-inference

Updated Jun 17, 2023
Jupyter Notebook

ray-project / llm-application

nlp scalable-machine-learning ray-distributed anyscale llm llm-serving

Updated Jun 14, 2023
Jupyter Notebook

LoopGlitch26 / Hinglish-AI-Mentor

Sponsor

Hinglish Chatbot powered by Azure Cognitive Services, Google Translate and Open AI

google azure nlp-machine-learning prompt-engineering llm-serving

Updated Jul 11, 2023
Jupyter Notebook

Stosan / commentator

generative-ai llm-serving

Updated Jul 5, 2023
Python

Improve this page

Add a description, image, and links to the llm-serving topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the llm-serving topic, visit your repo's landing page and select "manage topics."