[NeurIPS'23 Oral] Visual Instruction Tuning: LLaVA (Large Language-and-Vision Assistant) built towards GPT-4V level capabilities.

ChatGPT爆火，开启了通往AGI的关键一步，本项目旨在汇总那些ChatGPT的开源平替们，包括文本大模型、多模态大模型等，为大家提供一些便利

Effective prompting for Large Multimodal Models like GPT-4 Vision or LLaVA. 🔥

"Video-ChatGPT" is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.

👁️ + 💬 + 🎧 = 🤖 Curated list of top foundation and multimodal models! [Paper + Code]

Code/Data for the paper: "LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding"

LLaVA server (llama.cpp).

Aligning Large Multi-Modal Model with Robust Instruction Tuning

HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models

Embed arbitrary modalities (images, audio, documents, etc) into large language models.

Demo python script app to interact with llama.cpp server using whisper API, microphone and webcam devices.

Docker image for LLaVA: Large Language and Vision Assistant

This repository includes the official implementation of our paper "Sight Beyond Text: Multi-Modal Training Enhances LLMs in Truthfulness and Ethics"

Chat with large languages models about the contents of an image via this native desktop client for Windows, macOS, and Linux.

Kani extension for supporting vision-language models (VLMs). Comes with model-agnostic support for GPT-Vision and LLaVA.

Turn yourself into a Halloween-styled character and get an original roast with the power of AI.

LLaVA: Large Language and Vision Assistant | RunPod Serverless Worker

Chain of Images for Intuitively Reasoning

LLaVA-JP is a Japanese VLM trained by LLaVA method