multimodal

This project is a multi-modal AI voice assistant that uses LM Studio, OpenAI API or Claude Code, audio processing with WhisperModel, speech recognition, clipboard extraction, and image processing to respond to user prompts.

The official implement of Unified Reasoning Emotion Generalist OneEmo

Single-agent multi-modal system for insurance damage claim verification. #1 globally, HackerRank Orchestrate June 2026 (15,295 registrants). Anthropic, OpenAI, Groq, and Gemini support with a deterministic safety gate

Token-lean web microfetch for LLM agents: any URL → clean markdown via CLI, MCP server, and Claude Code plugin. Real browser-cookie auth, passkeys, anti-bot reach, on-by-default prompt-injection defense, plus on-device multimodal ASR/OCR. A single Rust binary — not a browser.

The code and data for "Summary-Oriented Vision Modeling for Multimodal Abstractive Summarization"

ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrieval

Dsh-visual-plugin.Give your text-only model eyes: forward user images to any OpenAI-compatible vision model and see the results in a Web UI right panel

LeManchot-Analysis is a system for abnormal detection in coupled visible-thermal images

[ACMMM'24] MoBA: Mixture of Bi-directional Adapter for Multi-modal Sarcasm Detection

An AI-driven character content generation system that implements a complete workflow from character creation, schedule planning, event planning, director output to video performance generation.

豆芽 Douya — 基于 llama.cpp 的隐私优先本地 AI 桌面助手 · 完全离线 · 多后端(CUDA/HIP/SYCL/Vulkan/OpenVINO/CPU) · GPU 加速 · 多模态 · RAG · OpenAI 兼容 API

The open source community's implementation of the all-new Multi-Modal Causal Attention from "DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention"

Project page for AV-Phys Bench: Do Joint Audio-Video Generation Models Understand Physics?

AOI: a model-agnostic perception layer giving computer-use agents adaptive video + audio perception with zero retraining, plus DynaCU-Bench (100 dynamic browser tasks). Paper + benchmark + code.

[WACV 2026 🔥] GAEA is a multimodal model with a new dataset and benchmark for context-aware image geolocation and QA.

Apache-2.0, model-agnostic macOS agent: bring your own model to deliver verified code, docs, slides, and computer actions.

A multimodal live AI assistant designed to enhance the browsing experience using Gemini.

Claude Code hook toolkit that gives vision-blind models a text description of pasted/tool-produced images.

Run NVIDIA's tri-modal Nemotron Omni (text+vision+audio) entirely on Apple Silicon — pure MLX runtime, parity-verified against NVIDIA's reference

Vision-Language Models on AMD GPUs — LLaVA, MiniGPT-4, Idefics on ROCm 🚀