multimodal

Code repository for "Post-pre-training for Modality Alignment in Vision-Language Foundation Models" (CVPR2025)

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

NeurIPS 2024 Track on Datasets and Benchmarks (Spotlight)

🚀 LLM-I: Transform LLMs into natural interleaved multimodal creators! ✨ Tool-use framework supporting image search, generation, code execution & editing

[WACV 2025] Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge

给 DeepSeek 补上「眼睛和耳朵」的多模态视觉插件：看图 / OCR / 物体检测 / 视频理解 / 语音转写 / 截图直读，一键安装（DSH 插件）。

TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models

Modular and Universal AI

Multimodal generative AI resources : talking heads, STT, TTS, image & video generation, and AI Agent

An efficient multi-modal instruction-following data synthesis tool and the official implementation of Oasis https://arxiv.org/abs/2503.08741.

[NeurIPS 2025] The official repository of "Inst-IT: Boosting Multimodal Instance Understanding via Explicit Visual Prompt Instruction Tuning"

TCyb 2018: Graph learning for multiview clustering

Pixel-native visual RAG ported to Rust on the ruvector ANN substrate (HNSW + IVF-Flat) — screenshot/document retrieval over visual embeddings, a Rust port of PixelRAG, with a metaharness benchmark CLI: npx rupixel

Implementation of the "Learn No to Say Yes Better" paper.

Easy wrapper for inserting LoRA layers in CLIP.

TMLR | This survey presents a comprehensive and structured synthesis of memory in LLMs and MLLMs, organizing the literature into a cohesive taxonomy comprising implicit, explicit, and agentic memory paradigms.

[NeurIPS 2025] A multimodal agent that can interact with its own PC in a multimodal manner.

🔥 Alternative to Ollama — multi-model serving with sub-ms model switching · CPU-only 20B inference for Edge AI · llama.cpp + stablediffusion.cpp

Finetune HunyuanImage 3.0, a 80B unified understanding and generation model

This repo contains evaluation code for the paper "MileBench: Benchmarking MLLMs in Long Context"