metrics

Real-time hardware and LLM inference monitoring — GPU, CPU, memory, and vLLM metrics streamed to a dashboard.

Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)

MirrorMetrics: How to evaluate Stable Diffusion LoRAs. A visual diagnostic tool to detect overfitting, check dataset quality, and fix training settings using InsightFace biometrics.

🐍📦 Ultra-fast Python package for calculating and analyzing the Word Error Rate (WER). Built for the scalable evaluation of speech and transcription accuracy.

Evals framework for Information Retrieval Systems