bytedance/UI-TARS-desktop
UI-TARS Desktop is a GUI Agent application that uses a Vision-Language Model to enable natural language control of computers across multiple platforms with real-time visual recognition and automation capabilities.
Awesome AI Agents › Desktop Automation
asweigart/pyautogui
PyAutoGUI is a cross-platform Python module designed for GUI automation, enabling users to programmatically control the mouse and keyboard. It simplifies the complex and technical details involved in automating user interface interactions across the three major operating systems: Windows, macOS, and Linux. PyAutoGUI provides a straightforward API that allows developers and users to simulate mouse movements, clicks, keyboard presses, and other input actions as if they were performed by a human. The module supports various functionalities such as moving the mouse to specific screen coordinates, clicking or double-clicking, dragging, typing text with customizable intervals, pressing individual keys, holding down keys, and executing hotkeys combinations. Additionally, PyAutoGUI offers features for displaying message boxes like alerts, confirmations, prompts, and password inputs, which can be useful for interactive scripts. It also integrates screenshot capabilities using the Pillow library, allowing users to capture the screen or locate images on the screen to interact with GUI elements visually. This image recognition feature helps automate tasks that require identifying buttons or other interface components by their appearance. The module is compatible with Python 2 and 3 and requires minimal dependencies depending on the operating system. For example, macOS requires pyobjc modules, Linux needs python-xlib, and Pillow must be installed for image-related functions. PyAutoGUI is primarily designed to work on the primary monitor and may have limited support for multi-monitor setups. Overall, PyAutoGUI is a powerful and user-friendly tool for automating repetitive tasks, testing applications, or creating custom scripts that interact with graphical user interfaces. Its ease of use, combined with comprehensive functionality for mouse and keyboard control, message dialogs, and screen capture, makes it a valuable resource for developers, testers, and automation enthusiasts.
https://github.com/asweigart/pyautogui
Also in Desktop Automation
bytebot-ai/bytebot
Bytebot is a containerized Linux desktop environment that enables programmatic control and remote access for automation, scraping, CI tasks, and remote work through a unified REST API and VNC interfaces.
OthersideAI/self-operating-computer
The Self-Operating Computer Framework enables multimodal AI models to autonomously operate a computer by interpreting screen visuals and executing mouse and keyboard actions to achieve user-defined objectives.
wonderwhy-er/DesktopCommanderMCP
Desktop Commander MCP is a server that integrates with the Claude desktop app to provide terminal control, file system search, and diff-based file editing capabilities, enabling advanced AI-assisted desktop management.
jacklandrin/OnlySwitch
OnlySwitch is a customizable all-in-one macOS menu bar app that consolidates system toggles, shortcuts, and AI-powered controls to enhance user productivity and convenience.
iflytek/astron-rpa
AstronRPA is an enterprise-grade open-source Robotic Process Automation desktop application that enables low-code/no-code workflow automation for desktop and web applications with strong enterprise security, collaboration, and AI agent integration.
yuruotong1/autoMate
autoMate is an AI-driven local automation assistant that uses natural language commands to enable users to automate complex computer tasks without programming, supporting local deployment and full interface control for enhanced privacy and flexibility.
TurixAI/TuriX-CUA
Open-source computer-use agent that performs real GUI actions on macOS and Windows desktops, with swappable vision-language model backends and MCP access for other agents.