multimodal

Improving Chest X-Ray Report Generation by Leveraging Warm-Starting

Awesome Multimodal Assistant is a curated list of multimodal chatbots/conversational assistants that utilize various modes of interaction, such as text, speech, images, and videos, to provide a seamless and versatile user experience.

Omni Model Benchmark with high quality and diversity, which reveals the Compositional Law. We’re now focused on Chinese scenarios — and actively seeking partners to co-build English & multilingual versions! Let’s expand global impact together.

Open agentic prompt-expansion harness for image and video generation, bridging polished demos, public APIs, and deployable workflows.

MCP server for OpenRouter — chat with 300+ LLMs (Claude, Gemini, GPT), analyze images / audio / video, generate images / speech / music / video (Veo 3.1, Sora, Seedance, Wan) from Claude Desktop, Cursor, Kiro, VS Code.

C++ framework to develop multimodal path planning requests

The largest multilingual image-text classification dataset. It contains fashion products.

Quick Long Video Understanding [TMLR2025]

An benchmark for evaluating the capabilities of large vision-language models (LVLMs)

A Comprehensive Survey on Evaluating Reasoning Capabilities in Multimodal Large Language Models.

Transparent proxy for Claude Code that auto-routes image-bearing requests to a multimodal model — so a non-multimodal primary model never crashes your long-running sessions.

[NeurIPS'25 Spotlight] Official implementation of "JavisGPT: A Unified Multi-modal LLM for Sounding-Video Comprehension and Generation"

My implementation of Kosmos2.5 from the paper: "KOSMOS-2.5: A Multimodal Literate Model"

一条命令，把微信变成任何 AI Agent 的入口 | Connect WeChat to any AI Agent with one command

A high-performance, universal serving framework for any-to-any models.

Image Classification Testing with LLMs

GroundVLP: Harnessing Zero-shot Visual Grounding from Vision-Language Pre-training and Open-Vocabulary Object Detection (AAAI 2024)

ARES - Automatic Robot Evaluation System. A simple, scalable solution for robotics research

The vision plugin for OpenCode that truly understands images. Inspect, read, and reason about any screenshot or picture with deeper understanding than any other plugin — fully local, private, and free.

Baselines for CCKS 2022 Task "Link Prediction for Multimodal Product Knowledge Graph"