raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK platform for observability, monitoring, and evaluation of AI agents and LLM projects, featuring tracing, debugging, advanced analytics, and project management tools.
Awesome AI Agents › Agent Monitoring Tools
Agent Monitoring Tools
MervinPraison/PraisonAI
PraisonAI is a production-ready Multi AI Agents framework that enables the creation and management of intelligent AI agents to automate and solve complex problems through collaboration and customization.
FailproofAI/failproofai
Local observability and policy-enforcement layer that hooks into 12 AI agent harnesses, including Claude Code, Codex and Hermes, to log every run and block dangerous tool calls before they execute.
apache/hertzbeat
Apache HertzBeat is an AI-powered open source real-time observability system that unifies metrics and logs collection, alerting, and notification with high performance and customizable monitoring capabilities.
junhoyeo/tokscale
Tokscale is a CLI utility and dashboard for tracking and visualizing token usage and costs across various AI coding agents.
matt1398/claude-devtools
Desktop and Docker application that reads local Claude Code session logs and reconstructs tool calls, thinking steps, subagent trees and per-turn token attribution in a visual interface.
eumemic/ai-legion
AI Legion is a platform for autonomous agents powered by large language models that collaborate to accomplish tasks with minimal human intervention.
traceroot-ai/traceroot
TraceRoot is open-source observability for AI agents that traces runs, detects failures with LLM judges, root-causes them against source code and GitHub history, opens fix PRs and evaluates each fix.
chiphuyen/sniffly
Sniffly is a local analytics dashboard that parses Claude Code logs to show usage statistics, error breakdowns and full message history, with optional shareable dashboards.
ZHangZHengEric/Sage
Sage is a production-ready, modular multi-agent system framework that intelligently decomposes and orchestrates complex tasks using advanced AI models and provides an interactive web interface for real-time task visualization.
agentscope-ai/OpenJudge
OpenJudge is an evaluation framework for AI agents and chatbots that ships 50+ validated graders, generates scenario rubrics and converts grading results into reward signals.
JudgmentLabs/judgeval
Open-source Python SDK from Judgment Labs that adds OpenTelemetry tracing and agent-judge evaluation to LLM applications, scoring live traffic and replaying historical traces to validate fixes.
jazzyalex/agent-sessions
Local-first macOS app that indexes coding-agent session histories from Codex, Claude Code, Cursor and others, adding search, transcript inspection, per-session quota accounting and resume.
luccast/crabwalk
Real-time companion monitor for OpenClaw (Clawdbot) agents that streams sessions, thinking states and tool calls from the gateway into a live ReactFlow node graph in the browser.
agentscope-ai/agentscope-studio
Local visualization and debugging toolkit for AgentScope applications, providing project and run management, a chat-style runtime view, OpenTelemetry tracing and evaluation analysis.
aiplanethub/openagi
OpenAGI is an open-source framework designed to enable the development of autonomous, human-like AI agents with long-term memory capabilities, applicable across various domains such as education, finance, and healthcare.
isekOS/ISEK
Isek is a decentralized agent network framework that enables intelligent, collaborative agent-to-agent systems powered by large language models, supporting autonomous peer discovery and distributed deployment for scalable and robust cooperation.
Azure-Samples/azure-ai-travel-agents
Azure AI Travel Agents is an enterprise application that uses multiple AI agents orchestrated by LlamaIndex.TS and MCP to enhance travel agency operations with features like customer query understanding, destination recommendation, and itinerary planning.
l3vels/team-of-ai-agents
L3AGI is an open-source framework that enables AI assistants to collaborate as effectively as human teams, providing tools for autonomous agents, data integration, and team management.
TheoLvs/westworld
Westworld is an experimental Python library for multi-agent simulation and optimization, inspired by Unity ML Agents, supporting various environments and agent interactions with features for visualization and reinforcement learning integration.
TheoLvs/westworldjs
Westworldjs is a JavaScript framework designed for creating and managing multi-agent simulations, enabling complex interactions among autonomous agents in a simulated environment.