raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK platform for observability, monitoring, and evaluation of AI agents and LLM projects, featuring tracing, debugging, advanced analytics, and project management tools.
Awesome AI Agents › Agent Monitoring Tools
eumemic/ai-legion
AI Legion is an autonomous agent platform powered by large language models (LLMs) designed to enable multiple agents to collaborate and accomplish tasks autonomously. The framework allows users to spin up several agents that can interact with each other and the user through a console interface. Each agent operates independently but can work collectively to achieve complex goals. The platform supports integration with OpenAI's GPT models, including GPT-3.5-turbo and GPT-4, providing flexibility in the choice of language model for different use cases. The setup process requires Node.js (version 10 or higher) and involves installing dependencies via npm. Users must configure environment variables with API keys for OpenAI and Google Custom Search to enable web search capabilities for the agents. The agents can perform actions and learn from their mistakes during operation, with mechanisms to handle errors such as incorrect action usage or formatting issues. This learning process helps agents improve their performance over time. Agent state is persistently stored in a directory structure that maintains memory, goals, and notes, allowing agents to resume tasks after restarts. This feature also facilitates debugging by enabling users to delete specific parts of an agent's memory or state to replay and fix issues. The platform encourages interactive development where users can request new features from agents, test them, and iterate based on agent feedback. Overall, AI Legion provides a robust environment for experimenting with autonomous AI agents that can collaborate, learn, and adapt to perform tasks with minimal human intervention. It is particularly useful for developers and researchers interested in multi-agent systems, autonomous AI workflows, and leveraging LLMs for complex task automation.
https://github.com/eumemic/ai-legion
Also in Agent Monitoring Tools
superset-sh/superset
Superset is a desktop application (macOS) designed to orchestrate and manage multiple CLI-based AI coding agents in parallel using isolated Git worktrees, featuring built-in diff viewing and IDE in...
MervinPraison/PraisonAI
PraisonAI is a production-ready Multi AI Agents framework that enables the creation and management of intelligent AI agents to automate and solve complex problems through collaboration and customization.
apache/hertzbeat
Apache HertzBeat is an AI-powered open source real-time observability system that unifies metrics and logs collection, alerting, and notification with high performance and customizable monitoring capabilities.
junhoyeo/tokscale
Tokscale is a CLI utility and dashboard for tracking and visualizing token usage and costs across various AI coding agents.
matt1398/claude-devtools
Desktop and Docker application that reads local Claude Code session logs and reconstructs tool calls, thinking steps, subagent trees and per-turn token attribution in a visual interface.
chiphuyen/sniffly
Sniffly is a local analytics dashboard that parses Claude Code logs to show usage statistics, error breakdowns and full message history, with optional shareable dashboards.
ZHangZHengEric/Sage
Sage is a production-ready, modular multi-agent system framework that intelligently decomposes and orchestrates complex tasks using advanced AI models and provides an interactive web interface for real-time task visualization.