xlang-ai/OSWorld
OSWorld is a benchmarking platform for evaluating multimodal AI agents performing open-ended tasks in real computer environments using virtual machines and various virtualization technologies.
Awesome AI Agents › Sensor Fusion Agents
google-deepmind/android_env
AndroidEnv is a Python library developed by DeepMind that transforms an Android device into a Reinforcement Learning (RL) environment. It enables researchers to define custom RL tasks on the Android Operating System, including any Android application, by allowing agents to interact with the device through a universal touchscreen action interface. Agents send localized touch and lift events to the system, which processes these inputs and returns pixel observations and rewards based on specific task definitions. This setup allows for a wide range of RL research in real-world contexts, as the environment simulates a real Android device running independently and unaltered, mirroring the experience of billions of users worldwide. The platform supports an almost infinite variety of tasks, from navigating Android games to performing actions like scrolling pages or sending emails, all sharing the same action interface. The environment operates in real-time, meaning the simulation speed cannot be accelerated, adding to the challenge. Observations are provided as RGB pixel values of the screen display, with options for downsampling. The action space is complex and unique, consisting of a hybrid continuous and discrete space that mimics touchscreen gestures such as tapping, scrolling, swiping, pinching, and drag & drop. These raw actions must often be composed into sequences to trigger meaningful changes, reflecting the natural interaction with Android devices. AndroidEnv also offers tools to modify the action interface for specific research needs, including discretization or hard-coding gesture skills. The library includes comprehensive documentation covering environment details, setup instructions, task definitions, and example tasks. It supports running on Unix-based systems primarily, with limited Windows support. The project is open-source and maintained by DeepMind, with a technical report and blog post available for further insights. It is a powerful platform for advancing RL research in complex, real-world-like environments using Android devices.
https://github.com/google-deepmind/android_env
Also in Sensor Fusion Agents
IBM/AssetOpsBench
Open IBM framework and benchmark for building, orchestrating and evaluating domain-specific AI agents in industrial asset operations, with MCP servers over sensors, failure modes, time series and work orders.
web-arena-x/webarena
WebArena is a self-hostable web environment designed for building and evaluating autonomous agents capable of realistic web navigation and interaction tasks.
google-research/android_world
AndroidWorld is a comprehensive environment and benchmark for autonomous agents to interact with and control Android devices through a live emulator, featuring diverse tasks and integration with web-based benchmarks.
karanchawla/GPS_IMU_Kalman_Filter
An Extended Kalman Filter implementation that fuses GPS, IMU, and encoder sensor data to accurately estimate the pose of a ground robot in a navigation frame.
OSU-NLP-Group/TravelPlanner
TravelPlanner is a benchmark for evaluating language agents in real-world travel planning tasks involving complex tool use and multiple constraints.
StonyBrookNLP/appworld
AppWorld is a high-fidelity execution environment simulating a world of apps and people to benchmark and evaluate interactive coding agents through diverse and challenging autonomous tasks.
nottelabs/open-operator-evals
Open-operator-evals is an open-source benchmarking framework that evaluates the performance of web operators and agents using multiple metrics and a reproducible dataset to provide transparent and statistically sound comparisons.