google-deepmind/acme
Acme is a flexible and scalable research framework providing modular reinforcement learning components and agents for developing and benchmarking RL algorithms.
Awesome AI Agents › Multimodal Model Benchmarks
david-abel/simple_rl
The project "simple_rl" is a straightforward and accessible framework designed for experimenting with Reinforcement Learning (RL) in Python. Its primary goals are simplicity and reproducibility of results, making it an excellent tool for both beginners and researchers who want to test RL algorithms without the complexity of more extensive libraries. The framework supports various RL agents, including basic ones like random agents, Q-learning agents, and R-Max agents, as well as more advanced implementations such as Q-learning with linear function approximation. It also includes implementations of planning algorithms like Value Iteration and Monte Carlo Tree Search (MCTS), although some features like MCTS are still under development. The framework provides a set of Markov Decision Processes (MDPs) and Object-Oriented MDPs (OO-MDPs) for testing, including classic environments like GridWorld, N-chain, and Taxi, as well as integration with OpenAI Gym environments. This allows users to experiment with a variety of tasks and scenarios. The project emphasizes reproducibility by introducing a feature that outputs experiment details to a file, enabling easy rerunning and verification of experiments. This feature currently supports MDPs but aims to extend to other models like POMDPs and Markov Games. The library requires dependencies such as numpy, matplotlib, and optionally pygame for visualizations. It supports both Python 2 and 3. The project includes utilities for charting and experiment management, and it encourages contributions to expand features like deep reinforcement learning (e.g., DQN), planning algorithms, efficiency improvements, and enhanced documentation. Overall, simple_rl is a valuable educational and research tool that balances ease of use with the capability to conduct meaningful RL experiments, making it suitable for those looking to learn or validate RL methods in a reproducible manner.
https://github.com/david-abel/simple_rl
Also in Multimodal Model Benchmarks
THUDM/AgentBench
AgentBench is a comprehensive benchmark platform designed to evaluate large language models as autonomous agents across diverse environments and tasks, facilitating research and development in LLM-based agent capabilities.
microsoft/WindowsAgentArena
Windows Agent Arena is a scalable platform for testing and benchmarking multi-modal AI agents in a realistic Windows OS environment using Azure ML for large-scale deployment.
WooooDyy/AgentGym
AgentGym is a versatile framework for developing, evaluating, and evolving large language model-based agents across diverse interactive environments with real-time feedback and scalability.
Ayanami0730/deep_research_bench
Benchmark and evaluation pipeline for deep research agents, scoring long-form generated reports on quality with the RACE framework and on citation trustworthiness with the FACT framework, plus a public leaderboard.
TheAgentCompany/TheAgentCompany
TheAgentCompany is a benchmarking platform that evaluates large language model agents on diverse real-world professional tasks within a simulated software company environment to assess their performance and impact on work-related activities.
eleurent/rl-agents
rl-agents is a comprehensive collection of reinforcement learning and planning algorithms with tools for experiment management, benchmarking, and performance monitoring in gym-compatible environments.
LehengTHU/Agent4Rec
Agent4Rec is a recommender system simulator using 1,000 LLM-powered generative agents initialized from MovieLens-1M to simulate realistic user interactions with personalized movie recommendations.