multimodal

自己阅读的多模态对话系统论文（及部分笔记）汇总

OpenAI-compatible bridge that exposes Codex CLI/SDK as /v1/chat/completions with multimodal + JSON schema

This repository provides core code for managing large volumes of video footage, enabling content understanding, automatic tagging, and vector database storage. It integrates multimodal models and LLMs for accurate descriptions and semantic search. A web interface allows visualization.

Ensemble Integration: a customizable pipeline for generating multi-modal, heterogeneous ensembles

The 1st place solution for SIGIR 2020 E-Commerce Workshop Multimodal Product Classification Challenge

✨ Official code for our paper: "Uncertainty-o: One Model-agnostic Framework for Unveiling Epistemic Uncertainty in Large Multimodal Models".

FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial Domain (EMNLP 2025)

Agent-based Multimodal Urban Moblity Simulator resulting from the ERC MAGnUM project

YesBut - Multimodal Satire Comprehension Dataset

模式识别课设代码：图文生成（CLIP+DALLE+BriVL）

A multimodal database engine written in Nim

[IEEE T-BIOM] FaceXBench: Evaluating Multimodal LLMs on Face Understanding

小红书图文笔记 Agent Skill，支持看图理解、主动提问、智能选题、实时热梗搜索、标题正文、标签评论钩子和风险检查。

Local LLM Discord Bot

Code for "MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching", TIP2025

The code for "VISTA: Enhancing Long-Duration and High-Resolution Video Understanding by VIdeo SpatioTemporal Augmentation" [CVPR2025]

Implementation of "With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition, BMVC, 2021" in PyTorch

ABC: Achieving Better Control of Multimodal Embeddings using VLMs [TMLR2025]

IA na Prática: LLM, RAG, MCP, Agents, Function Calling, Multimodal, TTS/STT e mais

MultimodalHugs is an extension of Hugging Face that offers a generalized framework for training, evaluating, and using multimodal AI models with minimal code differences, ensuring seamless compatibility with Hugging Face pipelines.