simular-ai/Agent-S
Agent S is an open-source framework that enables autonomous agents to interact with computers through GUIs, leveraging large language models and web retrieval to perform complex tasks with state-of-the-art performance.
Awesome AI Agents › Visual GUI Agents
agno-agi/agent-ui
Agent UI is a modern chat interface designed specifically for interacting with AI agents, providing a clean and user-friendly experience. It is built using a contemporary technology stack including Next.js, Tailwind CSS, and TypeScript, ensuring a responsive and customizable user interface. The project offers real-time streaming support, allowing users to see agent responses as they are generated, enhancing interactivity and engagement. One of the standout features of Agent UI is its support for tool calls, which visually represent the tools an AI agent uses during interactions along with their results. This transparency helps users understand the agent's actions and decisions. Additionally, the interface can display the agent's reasoning steps when available, providing insight into the thought process behind responses. This feature is particularly useful for complex queries where understanding the rationale is important. Agent UI also supports references, showing the sources used by the agent to generate its answers, which adds credibility and traceability to the information provided. The interface is multi-modal, capable of handling various content types such as images, videos, and audio, making it versatile for different use cases. The UI is highly customizable thanks to Tailwind CSS, allowing developers to easily adapt the look and feel to their needs. The project integrates other modern tools like shadcn/ui and Framer Motion to enhance the user experience with smooth animations and polished components. Setting up Agent UI is straightforward, with options for automatic installation via a command or manual setup by cloning the repository and installing dependencies. It connects by default to a local Agno Playground backend but can be configured to connect to other endpoints. The project is open for contributions and is licensed under the MIT License, encouraging community involvement and collaboration. Overall, Agent UI serves as a ready-to-use, modern, and extensible chat interface tailored for AI agents, making it easier for developers and users to interact with intelligent systems effectively.
https://github.com/agno-agi/agent-ui
Also in Visual GUI Agents
TencentQQGYLab/AppAgent
AppAgent is an LLM-based multimodal agent framework that enables AI agents to operate smartphone applications through human-like interactions such as tapping and swiping without backend access.
showlab/ShowUI
ShowUI is an open-source, end-to-end vision-language-action model designed for intelligent GUI agents and computer interaction, featuring advanced training, inference, and deployment capabilities.
zai-org/CogAgent
CogAgent is an advanced open-source vision-language model-based GUI agent supporting bilingual interaction and excelling in GUI perception, reasoning, and task generalization.
Ayush0Chaudhary/blurr
Panda is an on-device Android agent written in Kotlin that reads the screen through the Accessibility Service and drives app interfaces with taps, swipes and typing to carry out natural language commands.
likaixin2000/ScreenSpot-Pro-GUI-Grounding
ScreenSpot-Pro is a comprehensive project providing tools, datasets, and benchmarks for GUI grounding tailored to professional high-resolution computer use, advancing AI-driven human-computer interaction.
xlang-ai/aguvis
AGUVIS is a unified pure vision-based framework for autonomous GUI agents that operate across multiple platforms, leveraging a novel two-stage training pipeline and inner monologue for enhanced planning and reasoning.
OSU-NLP-Group/UGround
UGround is a universal visual grounding project for GUI agents enabling AI to navigate and interact with digital interfaces as humans do, achieving state-of-the-art results on multiple benchmarks.