bytedance/UI-TARS-desktop
UI-TARS Desktop is a GUI Agent application that uses a Vision-Language Model to enable natural language control of computers across multiple platforms with real-time visual recognition and automation capabilities.
Awesome AI Agents › Desktop Automation
showlab/assistgui
AssistGUI is a task-oriented desktop graphical user interface automation tool that leverages large language models (LLMs) to assist users in automating operations within desktop productivity software. The project is designed to enhance user efficiency by monitoring the user's screen and executing commands that control the mouse and keyboard. This allows for streamlined task execution and improved productivity by automating repetitive or complex GUI interactions. The system is capable of understanding user commands and translating them into precise actions on the desktop environment, making it a powerful assistant for various software applications. The project is developed by a team of researchers and engineers and aims to provide a seamless integration of AI-driven automation in everyday desktop tasks. Although the code is not yet publicly available, the project website offers additional information and updates. AssistGUI represents a significant advancement in the field of human-computer interaction by combining AI with GUI automation to create an intelligent assistant that can adapt to different software environments and user needs. The tool is particularly useful for users who require automation in productivity software, reducing manual effort and minimizing errors. Overall, AssistGUI is positioned as a cutting-edge solution for desktop automation, leveraging the capabilities of LLMs to interpret and execute user commands effectively.
https://github.com/showlab/assistgui
Also in Desktop Automation
asweigart/pyautogui
PyAutoGUI is a cross-platform Python module that enables programmatic control of the mouse and keyboard for GUI automation, including features like screen capture and image recognition.
bytebot-ai/bytebot
Bytebot is a containerized Linux desktop environment that enables programmatic control and remote access for automation, scraping, CI tasks, and remote work through a unified REST API and VNC interfaces.
OthersideAI/self-operating-computer
The Self-Operating Computer Framework enables multimodal AI models to autonomously operate a computer by interpreting screen visuals and executing mouse and keyboard actions to achieve user-defined objectives.
wonderwhy-er/DesktopCommanderMCP
Desktop Commander MCP is a server that integrates with the Claude desktop app to provide terminal control, file system search, and diff-based file editing capabilities, enabling advanced AI-assisted desktop management.
jacklandrin/OnlySwitch
OnlySwitch is a customizable all-in-one macOS menu bar app that consolidates system toggles, shortcuts, and AI-powered controls to enhance user productivity and convenience.
iflytek/astron-rpa
AstronRPA is an enterprise-grade open-source Robotic Process Automation desktop application that enables low-code/no-code workflow automation for desktop and web applications with strong enterprise security, collaboration, and AI agent integration.
yuruotong1/autoMate
autoMate is an AI-driven local automation assistant that uses natural language commands to enable users to automate complex computer tasks without programming, supporting local deployment and full interface control for enhanced privacy and flexibility.