multimodal-large-language-models

Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Pre-training Dataset and Benchmarks

[CVPR 2026] LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling

From scratch implementation of a vision language model in pure PyTorch

Official implementation of GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents

This repo contains evaluation code for the paper "BLINK: Multimodal Large Language Models Can See but Not Perceive". https://arxiv.org/abs/2404.12390 [ECCV 2024]

Awesome music generation model——MG²

[CVPR 2026] OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe

[ICLR 2025] AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark

(NIPS 2025) OpenOmni: Official implementation of Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Real-Time Self-Aware Emotional Speech Synthesis

A multimodal chat interface with many tools.

[ICCV 2025] Explore the Limits of Omni-modal Pretraining at Scale

Matryoshka Multimodal Models

[ICLR2025 Oral] ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understanding

Your SDK solves all of this. One interface. Unified logic. Local + hosted models. Fine-tuning. Agent tools. Enterprise-ready. Hybrid RAG.Star 🌟 if you like it!

Awesome Audio-Visual Intelligence, Survey of Audio-Visual Intelligence

A Comprehensive Survey on Evaluating Reasoning Capabilities in Multimodal Large Language Models.

[NeurIPS'25 Spotlight] Official implementation of "JavisGPT: A Unified Multi-modal LLM for Sounding-Video Comprehension and Generation"

Repository for awesome spatial/visual reasoning MLLMs. (focus more on embodied applications)

Sample Spring AI Application with several use cases

Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"