model-evaluation

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

an MLOps/LLMOps platform

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.

Ollama Model Test - Figure out the best model for the task

OpenLLM Monitor is a plug-and-play, real-time observability dashboard for monitoring and debugging LLM API calls across OpenAI, Ollama, OpenRouter, and more. Tracks tokens, latency, cost, retries, and lets you replay prompts — fully open-source and self-hostable.

ESGenius: EMNLP 2025 Main Oral benchmark for LLM ESG and sustainability knowledge, with 1,136 questions, evaluation code, interactive heatmap, and Hugging Face dataset release.

Madlab is an advanced AI development studio designed to streamline the process of dataset management, model training, and evaluation. It combines a robust Node.js/Express backend with a powerful Python-based training engine and a modern React frontend.

Embark on the "Reinforcement Learning from Human Feedback" course and align Large Language Models (LLMs) with human values.