最有价值项目

[🔥 ICLR 2026] - Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots

Elysium Knowledge Repository is an open source initiative to embed all of Humanity's multi-modal knowledge and wisdom.

Aster describes what a lecture video shows but never says, so blind and low-vision students can follow it

This project is a multi-modal AI voice assistant that uses LM Studio, OpenAI API or Claude Code, audio processing with WhisperModel, speech recognition, clipboard extraction, and image processing to respond to user prompts.

7.67× LoRA / 8.35× Full FT speedup for Qwen3.5 (0.8B–27B) on NVIDIA DGX Spark — wall-clock parity with rented H100. Lossless within BF16. Three-command interactive wizard handles model picker, data validator, training, and merge.

The official implement of Unified Reasoning Emotion Generalist OneEmo

Single-agent multi-modal system for insurance damage claim verification. #1 globally, HackerRank Orchestrate June 2026 (15,295 registrants). Anthropic, OpenAI, Groq, and Gemini support with a deterministic safety gate

Token-lean web microfetch for LLM agents: any URL → clean markdown via CLI, MCP server, and Claude Code plugin. Real browser-cookie auth, passkeys, anti-bot reach, on-by-default prompt-injection defense, plus on-device multimodal ASR/OCR. A single Rust binary — not a browser.

The code and data for "Summary-Oriented Vision Modeling for Multimodal Abstractive Summarization"

ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrieval

Dsh-visual-plugin.Give your text-only model eyes: forward user images to any OpenAI-compatible vision model and see the results in a Web UI right panel

LeManchot-Analysis is a system for abnormal detection in coupled visible-thermal images

[ACMMM'24] MoBA: Mixture of Bi-directional Adapter for Multi-modal Sarcasm Detection

An AI-driven character content generation system that implements a complete workflow from character creation, schedule planning, event planning, director output to video performance generation.

豆芽 Douya — 基于 llama.cpp 的隐私优先本地 AI 桌面助手 · 完全离线 · 多后端(CUDA/HIP/SYCL/Vulkan/OpenVINO/CPU) · GPU 加速 · 多模态 · RAG · OpenAI 兼容 API

The open source community's implementation of the all-new Multi-Modal Causal Attention from "DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention"

Project page for AV-Phys Bench: Do Joint Audio-Video Generation Models Understand Physics?

AOI: a model-agnostic perception layer giving computer-use agents adaptive video + audio perception with zero retraining, plus DynaCU-Bench (100 dynamic browser tasks). Paper + benchmark + code.

[WACV 2026 🔥] GAEA is a multimodal model with a new dataset and benchmark for context-aware image geolocation and QA.

Apache-2.0, model-agnostic macOS agent: bring your own model to deliver verified code, docs, slides, and computer actions.