OpenNSWM-Lab/FAROS
Blueprint-driven AutoResearch runtime that orchestrates an LLM research workflow from idea refinement through experiment scaffolding, LaTeX paper drafting and simulated peer review.
Awesome AI Agents › Research Platforms & Simulators
Purewhiter/mobilegym
MobileGym is a simulation platform for research on mobile GUI agents. Instead of driving real phones or heavyweight Android emulators, it re-implements mobile applications as browser-hosted simulations whose internal state is fully programmable and readable. The release ships 28 simulated apps and 416 parameterized task templates, each paired with a code-level judge that inspects application state directly, so success is verified deterministically in well under a millisecond rather than by screenshot comparison or an LLM grader. Because each instance is a lightweight browser context, the platform reports running 256 parallel instances on a single server at roughly 400 MB of memory and about three seconds of cold start per instance, which makes large-scale rollout collection practical. The repository contains the simulator itself, a benchmark suite of test tasks with a published leaderboard covering several vision-language agents, and online reinforcement learning training code under mobilegym-rl. The authors document a sim-to-real study in which a GRPO run on a small vision-language model gains a large margin in simulation and retains most of that gain when the trained policy is moved to a physical device. The codebase is split between TypeScript for the simulated applications and Python for the agent, evaluation and training side, and it documents how to add new apps and task templates so the environment can be extended. It targets researchers training or evaluating GUI agents who need reproducible, parallel and safe environments, and it comes with a paper preprint, a project site and a live demo.
https://github.com/Purewhiter/mobilegym
Also in Research Platforms & Simulators
WooooDyy/AgentGym-RL
Framework for training LLM agents on long-horizon, multi-turn decision making with reinforcement learning across a broad set of interactive environments, plus the ScalingInter-RL training method.
SimWorld-AI/SimWorld
SimWorld is an Unreal Engine 5 based open-ended simulator for developing and evaluating LLM and VLM agents in realistic physical and social environments such as procedurally generated cities.
Tencent/AICGSecEval
A.S.E is Tencent's repository-level benchmark that measures the security of AI-generated code, building tasks from real GitHub projects and CVE patches and scoring output with static and dynamic analysis.
facebookresearch/meta-agents-research-environments
Meta's research platform for evaluating AI agents in dynamic simulated environments, hosting the Gaia2 benchmark of 800 evolving scenarios across ten universes, with a CLI, a web GUI and a public leaderboard.
HUST-AI-HYZ/MemoryAgentBench
Benchmark and evaluation harness that measures memory in LLM agents across accurate retrieval, test-time learning, long-range understanding and conflict resolution using incremental multi-turn interactions.
DigiRL-agent/digirl
DigiRL is a research project providing code and resources for training autonomous reinforcement learning agents to control Android devices in real-world environments using novel training algorithms and multiple training modes.
xlang-ai/Spider2-V
Spider2-V is a NeurIPS 2024 research project providing a virtual machine environment and dataset to evaluate multimodal agents' ability to automate data science and engineering workflows using state-of-the-art vision-language models.