llm-evaluation-framework

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

The LLM Evaluation Framework

Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)