multimodal

The official implementation of 'Align and Attend: Multimodal Summarization with Dual Contrastive Losses' (CVPR 2023)

VideoRAG: Retrieval-Augmented Generation over Video Corpus

CLIP: Connecting Text and Image (Learning Transferable Visual Models From Natural Language Supervision)

A content-creation agent runtime for reproducible multimodal production — projects, workflows, skills, provenance.

Near-native image understanding for DeepSeek Harness

Retrieval of fully structured data made easy. Use LLMs or custom models. Specialized on PDFs and HTML files. Extensive support of tabular data extraction and multimodal queries.

OpenCode skill suite for MiniMax H3 directing, routing, multishot planning, prompt generation, and review.

[ICML 2024] The offical Implementation of "DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning"

This repo contains the code for "MEGA-Bench Scaling Multimodal Evaluation to over 500 Real-World Tasks" [ICLR 2025]

MmCows: A Multimodal Dataset for Dairy Cattle Monitoring

A Step-by-Step Implementation of Google Veo 3 Architecture from Scratch

Your computer was paying attention the whole time. 14-modality capture. Proactive intelligence. Computer-use training data. Native macOS. All on-device. Open source.

Open Scripts and pipelines from the Multimodal Imaging and Connectome Analysis Lab at the Montreal Neurological Institute

Gemma models running 100% local on an 8 GB Jetson Orin Nano: voice assistants with RAG, multi-agent NPCs, live EN-JA translation, and a Gemma 4 voice + vision agent that decides when to look through the webcam. The demos from the Gemma 2 launch at Google Tokyo.

ICML 2025 - Impossible Videos

Production-grade multimodal RAG for financial document intelligence. Chart understanding · hybrid retrieval · numeric guardrails · multi-tenancy · full observability.

Nomi enable people to use computer more simply.

[ICRA'23] Dataset of Moving Object Detection; Official Implementation of "RGB-Event Fusion for Moving Object Detection in Autonomous Driving"

Awesome Multimodal Assistant is a curated list of multimodal chatbots/conversational assistants that utilize various modes of interaction, such as text, speech, images, and videos, to provide a seamless and versatile user experience.

[NeurIPS 2024] Terra: A Multimodal Spatio-Temporal Dataset Spanning the Earth