multimodal

building various transformer model architectures and its modules from scratch.

Code release for "GAN-Tree: An Incrementally Learned Hierarchical Generative Framework for Multi-Modal Data Distributions", ICCV 2019

DeepSeek Harness 插件：DeepSeek Pro 大脑 + 自动识图。GUI 附加图片默认经官方 deepseek-v4-flash-vision-exp 原生识图，转译成文字后交给 DeepSeek 作答（纯文本的 V4-Pro 也能看图）；支持百炼/智谱/OpenRouter 等任意 OpenAI 兼容 VLM，无 key 自动探测本地 Ollama；安装时有一问式确认

Incorporate Image, Text and Tabular Data with HuggingFace Transformers

[CVPR 2026] Flow Matching for Multimodal Distributions

OMERO.web plugin for the Vitessce multimodal data viewer.

Multimodal document QA: vision + retrieval over PDFs (LLaVA + LlamaIndex)

🦫 Open-source iOS AI companion with a live VRM avatar that sees you — bring-your-own-key chat, vision, speech-to-text & text-to-speech (OpenAI, Anthropic, Gemini, ElevenLabs, Groq & more). SwiftUI.

📚 A curated list of Awesome Efficient dLLMs Papers with Codes

A production-grade multi-agent system using ReAct orchestration, MCP, A2A, and LMMs for hyper-local property valuation with Evals.

🧠 Multimodal Retrieval-Augmented Generation that "weaves" together text and images seamlessly. 🪡

TRIBE v2 — multimodal fMRI brain encoding model inference in Rust

Chinese-Multimodal-Sentiment-Analysis is an innovative project that integrates advanced deep learning techniques to analyze sentiments in Chinese language data. Utilizing a sophisticated blend of textual, audio, and visual modalities, this project offers a comprehensive approach to understanding and interpreting complex emotional expressions.

Run Ornith, Qwen, Nemotron & DeepSeek optimized on AMD Strix Halo — unified OpenAI-compatible server with tuned ROCmFP4 quants, validated 262K context, vision support & hot-swap model zoo (iGPU + NPU)

Pytorch code for "BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Digital Behavioural Change"

[EACL'23] COVID-VTS: Fact Extraction and Verification on Short Video Platforms

Repository to train multimodal latent reasoning tokens for Qwen 2.5 VL.

项目描述：为解决电商客服在物流、售后等高频咨询场景下，人工重复查询、知识定位慢及信息需反复核对等痛点，搭建 Agent 智能客服系统。通过意图分流、业务工具调用与 RAG 知识库检索，完成订单物流查询、售后信息收集和商品问答，提升响应效率与回答准确性。

Benchmarking Cross-Modal Learning from Naturalistic Egocentric Video Data

CLI, MCP server, and TypeScript library that extracts text, layout, and page images from PDFs for AI agents — when extraction goes wrong, it says so per page and says what to do next.