OpenNSWM-Lab/FAROS
Blueprint-driven AutoResearch runtime that orchestrates an LLM research workflow from idea refinement through experiment scaffolding, LaTeX paper drafting and simulated peer review.
Awesome AI Agents › Research Platforms & Simulators
ltzheng/Synapse
Synapse is an advanced agent designed for computer control tasks, introduced in the ICLR 2024 paper titled "Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control." The core innovation of Synapse lies in its use of trajectory-as-exemplar prompting combined with an associated memory system to enhance performance in solving complex computer control challenges. This approach allows the agent to leverage past trajectories as exemplars, effectively using memory to guide its actions and decisions. The project provides comprehensive code and resources to replicate the experiments and results presented in the paper. It supports environments like MiniWoB++ and Mind2Web, which are benchmarks for evaluating computer control agents. The repository includes detailed instructions for installation, setup, and running experiments, including building the exemplar memory using Faiss and preparing datasets from Mind2Web. Synapse demonstrates superior performance compared to state-of-the-art methods on both MiniWoB++ and Mind2Web benchmarks, as evidenced by the provided performance charts and experimental results. The project also supports fine-tuning capabilities, allowing users to improve the agent's performance further on the Mind2Web dataset using advanced models like CodeLlama with LoRA fine-tuning techniques. The repository is well-documented, providing clear guidance on environment setup, running experiments, fine-tuning, and evaluation. It also offers access to agent trajectories and updates through a dedicated website and downloadable resources. The project is licensed under MIT, ensuring open access and usability for research and development purposes. Overall, Synapse represents a significant advancement in the field of computer control through its novel use of trajectory-based prompting and memory integration, offering a robust framework for developing intelligent agents capable of complex task execution in simulated environments.
https://github.com/ltzheng/Synapse
Also in Research Platforms & Simulators
WooooDyy/AgentGym-RL
Framework for training LLM agents on long-horizon, multi-turn decision making with reinforcement learning across a broad set of interactive environments, plus the ScalingInter-RL training method.
Purewhiter/mobilegym
MobileGym is a browser-hosted, fully programmable Android-style simulation platform with 28 simulated apps and 416 task templates for benchmarking and reinforcement-learning training of mobile GUI agents.
SimWorld-AI/SimWorld
SimWorld is an Unreal Engine 5 based open-ended simulator for developing and evaluating LLM and VLM agents in realistic physical and social environments such as procedurally generated cities.
Tencent/AICGSecEval
A.S.E is Tencent's repository-level benchmark that measures the security of AI-generated code, building tasks from real GitHub projects and CVE patches and scoring output with static and dynamic analysis.
facebookresearch/meta-agents-research-environments
Meta's research platform for evaluating AI agents in dynamic simulated environments, hosting the Gaia2 benchmark of 800 evolving scenarios across ten universes, with a CLI, a web GUI and a public leaderboard.
HUST-AI-HYZ/MemoryAgentBench
Benchmark and evaluation harness that measures memory in LLM agents across accurate retrieval, test-time learning, long-range understanding and conflict resolution using incremental multi-turn interactions.
DigiRL-agent/digirl
DigiRL is a research project providing code and resources for training autonomous reinforcement learning agents to control Android devices in real-world environments using novel training algorithms and multiple training modes.