google-deepmind/acme
Acme is a flexible and scalable research framework providing modular reinforcement learning components and agents for developing and benchmarking RL algorithms.
Awesome AI Agents › Multimodal Model Benchmarks
eleurent/rl-agents
The rl-agents project is a comprehensive collection of Reinforcement Learning (RL) and Planning algorithms designed to facilitate experimentation and benchmarking in RL environments. It provides implementations of various RL agents, including value-based methods like Deep Q-Network (DQN) and Fitted-Q, as well as planning algorithms such as Value Iteration, Cross-Entropy Method, and Monte-Carlo Tree Search (MCTS) variants. The project supports safe planning and safe value-based approaches, including Robust Value Iteration and Budgeted Fitted-Q, which are crucial for applications requiring reliability and safety guarantees. The repository is structured to allow easy installation via pip and offers a command-line interface to run experiments, evaluate agents on different environments, and perform benchmarking with multiple agents and environments in parallel. The environments are compatible with OpenAI Gym, and agents follow a standard interface for interaction, making it straightforward to integrate new environments and agents. Monitoring tools are integrated to track agent performance, including logging, Gym Monitor statistics, and Tensorboard support for visualizing training progress and metrics. The project emphasizes reproducibility by saving metadata and configurations for each run. The planning algorithms cover a range of techniques from classical dynamic programming (Value Iteration) to advanced sampling-based methods (Cross-Entropy Method) and tree search strategies (MCTS and its variants like UCT, OPD, OLOP). Safe planning methods address robustness in uncertain environments. Value-based agents include popular deep RL methods, with configurations allowing customization of neural network architectures and exploration strategies. Overall, rl-agents is a versatile toolkit for researchers and practitioners in reinforcement learning, providing a rich set of algorithms, easy experiment management, and comprehensive monitoring capabilities to support the development and evaluation of RL agents in various settings.
https://github.com/eleurent/rl-agents
Also in Multimodal Model Benchmarks
THUDM/AgentBench
AgentBench is a comprehensive benchmark platform designed to evaluate large language models as autonomous agents across diverse environments and tasks, facilitating research and development in LLM-based agent capabilities.
microsoft/WindowsAgentArena
Windows Agent Arena is a scalable platform for testing and benchmarking multi-modal AI agents in a realistic Windows OS environment using Azure ML for large-scale deployment.
WooooDyy/AgentGym
AgentGym is a versatile framework for developing, evaluating, and evolving large language model-based agents across diverse interactive environments with real-time feedback and scalability.
Ayanami0730/deep_research_bench
Benchmark and evaluation pipeline for deep research agents, scoring long-form generated reports on quality with the RACE framework and on citation trustworthiness with the FACT framework, plus a public leaderboard.
TheAgentCompany/TheAgentCompany
TheAgentCompany is a benchmarking platform that evaluates large language model agents on diverse real-world professional tasks within a simulated software company environment to assess their performance and impact on work-related activities.
LehengTHU/Agent4Rec
Agent4Rec is a recommender system simulator using 1,000 LLM-powered generative agents initialized from MovieLens-1M to simulate realistic user interactions with personalized movie recommendations.
web-arena-x/visualwebarena
VisualWebArena is a benchmark for evaluating autonomous multimodal language agents on complex and realistic web-based visual tasks.