benchmark

This repo contains the code for "VLM2Vec / MMEB" [ICLR 2025], "VLM2Vec-V2 / MMEB-V2" [TMLR 2026], and "MMEB-V3" [COLM 2026]

Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表，主要面向基础大模型评测，旨在探求生成式AI的技术边界.

Open-source benchmark for browser AI agents on daily tasks.

Dataset and benchmark for RAG on company internal documents.

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

[ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI

MCPMark is a comprehensive, stress-testing MCP benchmark designed to evaluate model and agent capabilities in real-world MCP use.

Benchmarking Legal Knowledge of Large Language Models

A universal flight control tuning framework

Gym Electric Motor (GEM): An OpenAI Gym Environment for Electric Motors

LLM Benchmark for Throughput via Ollama (Local LLMs)

A long-horizon benchmark harness: give a coding agent a real game engine, professional conditions and time, and ask it to build an open-world game. Harness only, no results.

Framework for benchmarking vector search engines

Official code repository of < CBGBench: Fill in the Blank of Protein-Molecule Complex Binding Graph >

Open-source CLI, API, web UI, and reproducible benchmarks for probabilistic AI-generated image detection.

Your faithful, impartial partner for audio evaluation — know yourself, know your rivals. 真实评测，知己知彼。A unified benchmark framework for ASR/TTS/Audio Codec/audio LLM evaluation

Benchmarking synthetic data generation methods.

Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Pre-training Dataset and Benchmarks

Strix Halo guide for AMD Ryzen AI MAX+ 395 / Radeon 8060S local LLM setup and benchmarks: Ollama, llama.cpp, Vulkan/RADV, ROCm, GGUF, and raw evidence.

LLM 并发性能测试工具，支持自动化压力测试和性能报告生成。