simular-ai/Agent-S
Agent S is an open-source framework that enables autonomous agents to interact with computers through GUIs, leveraging large language models and web retrieval to perform complex tasks with state-of-the-art performance.
Awesome AI Agents › Visual GUI Agents
Visual GUI Agents
TencentQQGYLab/AppAgent
AppAgent is an LLM-based multimodal agent framework that enables AI agents to operate smartphone applications through human-like interactions such as tapping and swiping without backend access.
showlab/ShowUI
ShowUI is an open-source, end-to-end vision-language-action model designed for intelligent GUI agents and computer interaction, featuring advanced training, inference, and deployment capabilities.
agno-agi/agent-ui
Agent UI is a modern, customizable chat interface for AI agents featuring real-time streaming, tool call visualization, reasoning steps, and multi-modal content support, built with Next.js and Tailwind CSS.
zai-org/CogAgent
CogAgent is an advanced open-source vision-language model-based GUI agent supporting bilingual interaction and excelling in GUI perception, reasoning, and task generalization.
Ayush0Chaudhary/blurr
Panda is an on-device Android agent written in Kotlin that reads the screen through the Accessibility Service and drives app interfaces with taps, swipes and typing to carry out natural language commands.
likaixin2000/ScreenSpot-Pro-GUI-Grounding
ScreenSpot-Pro is a comprehensive project providing tools, datasets, and benchmarks for GUI grounding tailored to professional high-resolution computer use, advancing AI-driven human-computer interaction.
xlang-ai/aguvis
AGUVIS is a unified pure vision-based framework for autonomous GUI agents that operate across multiple platforms, leveraging a novel two-stage training pipeline and inner monologue for enhanced planning and reasoning.
OSU-NLP-Group/UGround
UGround is a universal visual grounding project for GUI agents enabling AI to navigate and interact with digital interfaces as humans do, achieving state-of-the-art results on multiple benchmarks.
eric-ai-lab/Screen-Point-and-Read
This project introduces the Tree-of-Lens agent for layout-aware screen reading of graphical user interfaces based on user-indicated points, enhancing GUI understanding and navigation verification.