bytedance/UI-TARS-desktop
UI-TARS Desktop is a GUI Agent application that uses a Vision-Language Model to enable natural language control of computers across multiple platforms with real-time visual recognition and automation capabilities.
Awesome AI Agents › Desktop Automation
yuruotong1/autoMate
autoMate is an AI-powered local automation tool designed to transform how users interact with their computers by automating repetitive tasks through natural language commands. Unlike traditional Robotic Process Automation (RPA) tools that often require complex setups and programming knowledge, autoMate leverages large language models and advanced AI technologies to understand user requirements expressed in everyday language and autonomously execute tasks. The project is built on OmniParser and integrates AI-driven capabilities such as intelligent screen content comprehension, simulating human vision and operations, and making autonomous decisions based on task needs. This allows the tool to operate on any visual interface, not limited to specific software, providing full interface control and flexibility. The tool supports local deployment, ensuring data security and privacy, which is a critical feature for users concerned about cloud-based solutions. It is designed to be user-friendly with no-code automation, enabling users without programming skills to describe tasks naturally and have the computer perform them automatically. The installation process is simplified, with support for Chinese environments and one-click deployment options. The project is still in early development stages, with ongoing rapid iteration and integration of the latest technologies. The developers actively share their research, challenges, and solutions related to AI and RPA in a dedicated community platform. autoMate supports several large language models, including OpenAI's GPT-4 series and the yeka proxy for users in regions with restricted access to OpenAI. The tool requires a computer with a dedicated NVIDIA GPU for optimal performance due to its reliance on OCR and visual annotation, which are resource-intensive. The project encourages community contributions to enhance features, fix bugs, and improve documentation, aiming to create a more intelligent and efficient future for computer automation. Overall, autoMate represents a significant step forward in making AI-driven automation accessible and practical for everyday computer users.
https://github.com/yuruotong1/autoMate
Also in Desktop Automation
asweigart/pyautogui
PyAutoGUI is a cross-platform Python module that enables programmatic control of the mouse and keyboard for GUI automation, including features like screen capture and image recognition.
bytebot-ai/bytebot
Bytebot is a containerized Linux desktop environment that enables programmatic control and remote access for automation, scraping, CI tasks, and remote work through a unified REST API and VNC interfaces.
OthersideAI/self-operating-computer
The Self-Operating Computer Framework enables multimodal AI models to autonomously operate a computer by interpreting screen visuals and executing mouse and keyboard actions to achieve user-defined objectives.
wonderwhy-er/DesktopCommanderMCP
Desktop Commander MCP is a server that integrates with the Claude desktop app to provide terminal control, file system search, and diff-based file editing capabilities, enabling advanced AI-assisted desktop management.
jacklandrin/OnlySwitch
OnlySwitch is a customizable all-in-one macOS menu bar app that consolidates system toggles, shortcuts, and AI-powered controls to enhance user productivity and convenience.
iflytek/astron-rpa
AstronRPA is an enterprise-grade open-source Robotic Process Automation desktop application that enables low-code/no-code workflow automation for desktop and web applications with strong enterprise security, collaboration, and AI agent integration.
TurixAI/TuriX-CUA
Open-source computer-use agent that performs real GUI actions on macOS and Windows desktops, with swappable vision-language model backends and MCP access for other agents.