An ecosystem of Rust libraries for working with large language models

Stable Diffusion in pure C/C++

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.

Run inference on MPT-30B using CPU

Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)

Large Language Models for All, 🦙 Cult and More, Stay in touch !

Self-evaluating interview for AI coders

🛡️Build LLM applications safely and reliably🛡️

CLIP inference in plain C/C++ with no extra dependencies

speech recognition in dart support all audio format and support server side client side, + support all language, only support in cpu only

Chat with your data privately using MPT-30b

Run inference on replit-3B code instruct model using CPU

WIP Library Text To Speech From Suno AI's Bark in C/C++ for fast inference

llama and other large language models on iOS and MacOS offline using GGML library.

🪶 Lightweight OpenAI drop-in replacement for Kubernetes

Making offline AI models accessible to all types of edge devices.

JavaScript bindings for the ggml-js library

LLaMAZoo - Platform for serving LLaMA models on your PC / Mac

Finetuning instruct-LLaMA on german datasets.