agentops

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

Evaluation and Tracking for LLM Experiments and AI Agents

A database of SDKs, frameworks, libraries, and tools for creating, monitoring, debugging and deploying autonomous AI agents

CrewAI agents that gather and analyze YouTube comments to generate insights to inform better content creation.

Geniusrise: Framework for building geniuses

A radically simple, reliable, and high performance template to enable you to quickly get set up building multi-agent applications