🔮 Multimodal AI services & pipelines with cloud-native stack: gRPC, Kubernetes, Docker, OpenTelemetry, Prometheus, Jaeger, etc.

<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.

Operating LLMs in production

Build Production-Grade AI Applications

A high-throughput and memory-efficient inference and serving engine for LLMs

AGiXT is a dynamic AI Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.

🏕️ Reproducible development environment for AI/ML (LLMOps)

Your open-source LLM experimentation, response validation and monitoring toolkit.

An awesome & curated list of best LLMOps tools for developers

ML Observability in a Notebook - Uncover Insights, Surface Problems, Monitor, and Fine Tune your Generative LLM, CV and Tabular Models

🐢 The testing framework for ML models, from tabular to LLMs

An open-source visual programming environment for LLM experimentation and prompt evaluation.

A reactive runtime for building durable AI agents

A scalable general purpose micro-orchestration framework for defining dataflows. You can use it to build dataframes, numpy matrices, python objects, ML models, LLM workflows, etc. Embed Hamilton anywhere python runs, e.g. spark, airflow, jupyter, fastapi, python scripts, etc. Comes with lineage out of the box.

FastAPI framework to build production-grade LLM applications

🧠 Motorhead is a memory and information retrieval server for LLMs.

Test your prompts. Evaluate and compare LLM outputs, catch regressions, and improve prompt quality.

Ray Aviary - evaluate multiple LLMs easily

本项目旨在分享大模型相关技术原理以及实战经验。

LLM Prompt Injection Detector