multimodal-large-language-models

Mobile-Agent: The Powerful GUI Agent Family

StarVector is a foundation model for SVG generation that transforms vectorization into a code generation task. Using a vision-language modeling architecture, StarVector processes both visual and textual inputs to produce high-quality SVG code with remarkable precision.

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

A cross-platform video structuring (video analysis) framework based on CV models & mLLM.

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

A family of lightweight multimodal models.

Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey

NEO Series: Native Vision-Language Models from First Principles

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.

(Accepted by IJCV) Liquid: Language Models are Scalable and Unified Multi-modal Generators

LLaVA-Mini is a unified large multimodal model (LMM) that can support the understanding of images, high-resolution images, and videos in an efficient manner.

Official code of "EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Model"

[NeurIPS 2025 Spotlight] Official implementation of Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.

Research Trends in LLM-guided Multimodal Learning.

[ICLR 2026] VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning

A curated list of awesome Multimodal studies.

[WACV 2024 Survey Paper] Multimodal Large Language Models for Autonomous Driving

LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM