bytedance/UI-TARS-desktop
UI-TARS Desktop is a GUI Agent application that uses a Vision-Language Model to enable natural language control of computers across multiple platforms with real-time visual recognition and automation capabilities.
Awesome AI Agents › Desktop Automation
nut-tree/nut.js
nut.js is a cross-platform native UI automation and testing tool designed to facilitate native UI interactions through keyboard and mouse controls. It also supports screen navigation based on image matching, making it a versatile toolkit for automating user interface tasks. The project is open source and free to use, with ongoing maintenance and development supported by a community of contributors and sponsors. nut.js emphasizes community involvement and sustainability in open source development. The toolkit offers a wide range of modules that cover various aspects of UI automation. These include clipboard operations like copying and pasting text, keyboard support for standard US layouts and multimedia keys, and mouse functionalities such as movement, scrolling, configurable speed, and dragging. Window management features allow listing, focusing, resizing, repositioning, minimizing, restoring, and inspecting GUI elements of windows. Screen-related capabilities include retrieving color information, highlighting regions, and finding images or text on the screen using additional provider packages. nut.js also supports hooks to trigger actions based on detected screen elements. Integration with popular tools like Jest and Electron is available, along with custom logging options. The project provides comprehensive documentation, tutorials, and API references on its website nutjs.dev, along with a Discord community for support and discussion. It requires certain platform-specific prerequisites, such as Xcode command line tools and permissions on macOS, and libXtst on Linux. nut.js is suitable for developers looking to automate UI testing or control native applications programmatically across different operating systems.
https://github.com/nut-tree/nut.js
Also in Desktop Automation
asweigart/pyautogui
PyAutoGUI is a cross-platform Python module that enables programmatic control of the mouse and keyboard for GUI automation, including features like screen capture and image recognition.
bytebot-ai/bytebot
Bytebot is a containerized Linux desktop environment that enables programmatic control and remote access for automation, scraping, CI tasks, and remote work through a unified REST API and VNC interfaces.
OthersideAI/self-operating-computer
The Self-Operating Computer Framework enables multimodal AI models to autonomously operate a computer by interpreting screen visuals and executing mouse and keyboard actions to achieve user-defined objectives.
wonderwhy-er/DesktopCommanderMCP
Desktop Commander MCP is a server that integrates with the Claude desktop app to provide terminal control, file system search, and diff-based file editing capabilities, enabling advanced AI-assisted desktop management.
jacklandrin/OnlySwitch
OnlySwitch is a customizable all-in-one macOS menu bar app that consolidates system toggles, shortcuts, and AI-powered controls to enhance user productivity and convenience.
iflytek/astron-rpa
AstronRPA is an enterprise-grade open-source Robotic Process Automation desktop application that enables low-code/no-code workflow automation for desktop and web applications with strong enterprise security, collaboration, and AI agent integration.
yuruotong1/autoMate
autoMate is an AI-driven local automation assistant that uses natural language commands to enable users to automate complex computer tasks without programming, supporting local deployment and full interface control for enhanced privacy and flexibility.