THUDM/AgentBench
AgentBench is a comprehensive benchmark platform designed to evaluate large language models as autonomous agents across diverse environments and tasks, facilitating research and development in LLM-based agent capabilities.
Awesome AI Agents › Multimodal Model Benchmarks
google-deepmind/acme
Acme is a comprehensive research framework designed for reinforcement learning (RL) that provides a library of modular RL components and agents. It aims to offer simple, efficient, and readable implementations of RL agents that serve as both reference models and strong baselines for algorithm performance. The framework is built to support flexibility and extensibility, allowing researchers to use the baseline agents as starting points for novel research. Acme's design supports running agents at various scales, from single-stream to distributed setups, making it suitable for a wide range of RL research scenarios. The project includes detailed code examples and notebooks to help users quickly get started with different agents and environments. These resources demonstrate how to instantiate and run agents, providing practical insights into the framework's usage. Acme supports integration with popular machine learning libraries such as JAX and TensorFlow, which are required to run the included agents. Additionally, it offers compatibility with several environments like OpenAI Gym, DeepMind Control Suite, and bsuite, facilitating diverse experimental setups. Installation instructions emphasize the use of Python virtual environments to manage dependencies and avoid conflicts. Users can install the core library via pip, with optional extras for JAX, TensorFlow, testing, and environments. The project is actively maintained by DeepMind researchers and is intended primarily for research purposes, with a focus on robustness and ongoing improvements. Acme is well-documented, with extensive technical reports and papers available to provide background and design rationale. It is a valuable tool for researchers looking to develop, test, and benchmark reinforcement learning algorithms in a flexible and scalable manner. The project encourages citation of its accompanying technical report when used in academic work, highlighting its role in advancing RL research.
https://github.com/google-deepmind/acme
Also in Multimodal Model Benchmarks
microsoft/WindowsAgentArena
Windows Agent Arena is a scalable platform for testing and benchmarking multi-modal AI agents in a realistic Windows OS environment using Azure ML for large-scale deployment.
WooooDyy/AgentGym
AgentGym is a versatile framework for developing, evaluating, and evolving large language model-based agents across diverse interactive environments with real-time feedback and scalability.
Ayanami0730/deep_research_bench
Benchmark and evaluation pipeline for deep research agents, scoring long-form generated reports on quality with the RACE framework and on citation trustworthiness with the FACT framework, plus a public leaderboard.
TheAgentCompany/TheAgentCompany
TheAgentCompany is a benchmarking platform that evaluates large language model agents on diverse real-world professional tasks within a simulated software company environment to assess their performance and impact on work-related activities.
eleurent/rl-agents
rl-agents is a comprehensive collection of reinforcement learning and planning algorithms with tools for experiment management, benchmarking, and performance monitoring in gym-compatible environments.
LehengTHU/Agent4Rec
Agent4Rec is a recommender system simulator using 1,000 LLM-powered generative agents initialized from MovieLens-1M to simulate realistic user interactions with personalized movie recommendations.
web-arena-x/visualwebarena
VisualWebArena is a benchmark for evaluating autonomous multimodal language agents on complex and realistic web-based visual tasks.