llava

[ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

Official Repository of paper VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding

AI Device Template Featuring Whisper, TTS, Groq, Llama3, OpenAI and more

Code/Data for the paper: "LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding"

llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2

From scratch implementation of a vision language model in pure PyTorch

A Ruby gem for interacting with Ollama's API that allows you to run open source AI LLMs (Large Language Models) locally.

Embed arbitrary modalities (images, audio, documents, etc) into large language models.

LLaVA server (llama.cpp).

Matryoshka Multimodal Models

[ICCV 2025] The official code of the paper "Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate".

中文医学多模态大模型 Large Chinese Language-and-Vision Assistant for BioMedicine

Self-host a ChatGPT-style web interface for Ollama 🦙

[NAACL 2024] MMC: Advancing Multimodal Chart Understanding with LLM Instruction Tuning

Open-source local AI video analyzer powered by Ollama. Visual search, automatic highlights, scene/action/object detection, audio analysis, and subtitle generation. Free, offline alternative to Twelve Labs, Runway, and Descript.

Image Classification Testing with LLMs

Give your computer an AI Brain

使用LLaMA-Factory微调多模态大语言模型的示例代码 Demo of Finetuning Multimodal LLM with LLaMA-Factory

RLLaVA is a user-friendly framework for multi-modal RL research and optimized for resource-constrained teams.

This repository contains a web application designed to execute relatively compact, locally-operated Large Language Models (LLMs).