video-understanding

OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

[ECCV2024] Video Foundation Models & Data for Multimodal Understanding

NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.

TeleMem is a high-performance drop-in replacement for Mem0, featuring semantic deduplication, long-term dialogue memory, and multimodal video reasoning.

Video understanding and self-verification for AI agents. Turn videos, streams, and agent screen recordings into searchable, timestamped evidence—then use THE LOOP to inspect, fix, and verify the work. MCP, CLI, REST, local-first.

[ICLR 2025] AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark

[ICLR 2026] DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learning

[ECCV 2022] Official Pytorch Implementation of the paper : " Zero-Shot Temporal Action Detection via Vision-Language Prompting "

[CVPR 2026 Highlight] WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning

Open-source local AI video analyzer powered by Ollama. Visual search, automatic highlights, scene/action/object detection, audio analysis, and subtitle generation. Free, offline alternative to Twelve Labs, Runway, and Descript.

ICML 2025 - Impossible Videos

[NeurIPS 2025] Deep Memory Backtracking for Long Video Understanding

OmniAgent (ICML 2026): the first native omni-modal agent for active video perception — a 7B agent that beats Qwen2.5-VL-72B with 73% fewer frames on LVBench.

Graph Distillation for Action Detection

ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model

TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models

Code for LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videos

Ask your video library anything — an AI agent that watches, reasons in a tool-use loop, and answers with playable clips and charts.