benchmark

Code for Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense? [COLM 2024]

KAREN: Unifying Hatespeech Detection and Benchmarking

[ICML'26] Toward Human-like Audio-Visual Intelligence of Omni-MLLMs

H.E.I.M.D.A.L.L looks at fleet telemetry and gives you natural-language insights. GPU data loading (cuDF), local LLM inference (Gemma 2), and production NIM on GKE. Open the notebooks, run cells, get answers! Quick start should not take longer than 10 minutes and the T4 path is completely free!

Vector Search Benchmarking suite

A comprehensive survey of Vision–Language Models: Pretrained models, fine-tuning, prompt engineering, adapters, and benchmark datasets

ESGenius: EMNLP 2025 Main Oral benchmark for LLM ESG and sustainability knowledge, with 1,136 questions, evaluation code, interactive heatmap, and Hugging Face dataset release.

We introduce a large semi-automatically generated dataset of ~400,000 descriptive sentences about commonsense knowledge that can be true or false in which negation is present in about 2/3 of the corpus in different forms that we use to evaluate LLMs

Cognitive-structured Multimodal Agent (CMA-Harness): a memory-centric agent for long-horizon multimodal understanding, generation, and editing — externalizing visual history into episodic memory with selective retrieval. Includes the M2CA-Bench benchmark.

An Interactive Game-based Vision Planning benchmark

Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026.

This project implements 30+ variants of ANN algorithms to find the K nearest neighbors in high-dimensional vector spaces. It is meant as a convenient sandbox: drop in your own ANN code, run a one-liner, and instantly compare build/search speed and recall against the bundled baselines.

AI hardware fit calculator for LLM, embedding, reranker, OCR and VLM workloads — VRAM, throughput, licensing and multi-GPU planning

[ACL 2026 Oral] Official code, UIQ benchmark, and pretrained checkpoints for "Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval" (Yoo et al., Sogang University)

WMB-100K — The first 100,000-turn benchmark for AI memory systems

Pytorch code for "BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Digital Behavioural Change"

EgoNormia | Benchmarking Physical Social Norm Understanding in VLMs

A curated list of papers, benchmarks, project pages, and code for proactive agents.

Benchmark OpenAI-compatible AI endpoints and AI Accelerators in a reproducible structured way

Project page for AV-Phys Bench: Do Joint Audio-Video Generation Models Understand Physics?