mllm

AUITestAgent is the first automatic, natural language-driven GUI testing tool for mobile apps, capable of fully automating the entire process of GUI interaction and function verification.

[CVPR 2026] LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling

A CPU Realtime VLM in 500M. Surpassed Moondream2 and SmolVLM. Training from scratch with ease.

Official code for Paper "Mantis: Multi-Image Instruction Tuning" [TMLR 2024 Best Paper]

mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video (ICML 2023)

This repository collects papers on VLLM applications. We will update new papers irregularly.

[ICCV25 Oral] Token Activation Map to Visually Explain Multimodal LLMs

[CVPR 2026] OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe

中文医学多模态大模型 Large Chinese Language-and-Vision Assistant for BioMedicine

【AIGC 实战入门笔记 —— AIGC 摩天大楼】分享 大语言模型（LLMs），大模型高效微调（SFT）,检索增强生成（RAG），智能体（Agent），PPT自动生成, 角色扮演，文生图（Stable Diffusion） ，图像文字识别（OCR），语音识别（ASR），语音合成（TTS），人像分割（SA），多模态（VLM），Ai 换脸(Face Swapping), 文生视频(VD)，图生视频（SVD），Ai 动作迁移，Ai 虚拟试衣，数字人，全模态理解（Omni），Ai音乐生成 干货学习 等 实战与经验。

[NeurIPS'25 Spotlight] Official implementation of "JavisGPT: A Unified Multi-modal LLM for Sounding-Video Comprehension and Generation"

[NeurIPS 2025] Deep Memory Backtracking for Long Video Understanding

Evaluation framework for paper "VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?"

OmniAgent (ICML 2026): the first native omni-modal agent for active video perception — a 7B agent that beats Qwen2.5-VL-72B with 73% fewer frames on LVBench.

R1-Track: Direct Application of MLLMs to Visual Object Tracking via Reinforcement Learning.

使用LLaMA-Factory微调多模态大语言模型的示例代码 Demo of Finetuning Multimodal LLM with LLaMA-Factory

[CVPR 2025] Augmenting Multimodal LLMs with Self-Reflective Tokens for Knowledge-based Visual Question Answering

在千问最新的多模态image-text模型Qwen3-VL-4B-Instruct 进行多种lora微调对比效果，通过langchain+RAG+多智能体(Multi-Agent)进行部署

ElasticMM: Elastic and Efficient MLLM Serving System

An efficient multi-modal instruction-following data synthesis tool and the official implementation of Oasis https://arxiv.org/abs/2503.08741.