raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK platform for observability, monitoring, and evaluation of AI agents and LLM projects, featuring tracing, debugging, advanced analytics, and project management tools.
Awesome AI Agents › Agent Monitoring Tools
apache/hertzbeat
Apache HertzBeat is an AI-powered next-generation open source real-time observability system designed to provide unified metrics and logs collection, centralized alerting distribution, intelligent management, and analysis. It operates without requiring an agent, supports high-performance clustering, and offers powerful custom monitoring and status page building capabilities. The platform integrates collection, analysis, alerting, and notification into a single system, enhanced with AI-powered interactions and features under HertzBeat AI, along with built-in MCP Server capabilities. It supports a unified metrics platform that is agentless and Prometheus-compatible, covering application services, programs, databases, caches, operating systems, big data, middleware, web servers, cloud-native environments, networks, and custom monitoring. The unified logging platform integrates multiple log sources through the OTLP protocol for reporting. The alerting platform consolidates internal alerts with various external alert sources, providing unified alert processing and analysis with flexible real-time and periodic threshold rules, grouping convergence, silence, and suppression features. Alerts are distributed through multiple channels including Email, Discord, Slack, Telegram, DingTalk, WeChat, FeiShu, SMS, Webhook, and ServerChan. The system allows configuration of protocols such as Http, Jmx, Ssh, Snmp, Jdbc, and Prometheus via online YML template files, enabling quick adaptation to new monitoring types like Kubernetes or Docker. It supports horizontal expansion of multi-collector clusters, multi-isolated network monitoring, and cloud-edge collaboration. Additionally, it offers powerful status page building capabilities to communicate real-time service status to users. The architecture supports various metric collection types defined as YML templates, encouraging community contributions for customized monitoring templates. Overall, Apache HertzBeat aims to help users quickly and conveniently achieve comprehensive observability requirements with AI intelligence, powerful customization, multi-type support, high performance, and easy scalability.
https://github.com/apache/hertzbeat
Also in Agent Monitoring Tools
superset-sh/superset
Superset is a desktop application (macOS) designed to orchestrate and manage multiple CLI-based AI coding agents in parallel using isolated Git worktrees, featuring built-in diff viewing and IDE in...
MervinPraison/PraisonAI
PraisonAI is a production-ready Multi AI Agents framework that enables the creation and management of intelligent AI agents to automate and solve complex problems through collaboration and customization.
junhoyeo/tokscale
Tokscale is a CLI utility and dashboard for tracking and visualizing token usage and costs across various AI coding agents.
matt1398/claude-devtools
Desktop and Docker application that reads local Claude Code session logs and reconstructs tool calls, thinking steps, subagent trees and per-turn token attribution in a visual interface.
eumemic/ai-legion
AI Legion is a platform for autonomous agents powered by large language models that collaborate to accomplish tasks with minimal human intervention.
chiphuyen/sniffly
Sniffly is a local analytics dashboard that parses Claude Code logs to show usage statistics, error breakdowns and full message history, with optional shareable dashboards.
ZHangZHengEric/Sage
Sage is a production-ready, modular multi-agent system framework that intelligently decomposes and orchestrates complex tasks using advanced AI models and provides an interactive web interface for real-time task visualization.