wandb/openui
OpenUI is an open-source tool that enables users to describe user interfaces using natural language and see them rendered live, leveraging large language models to generate UI components across multiple frontend frameworks.
Awesome AI Agents › GUI Action Mapping
sullyo/prompt2ui
The project 'prompt2ui' by sullyo is a web application built using the Next.js framework, designed to convert prompts into user interfaces for fun and experimentation. It is a Next.js project bootstrapped with create-next-app, which provides a modern React-based framework for building server-side rendered and statically generated web applications. The project allows users to run a development server locally and see the results in their browser at localhost:3000. The main page of the application can be edited and auto-updates as changes are made, facilitating rapid development and testing. The project uses next/font for font optimization, specifically loading the Inter font from Google Fonts to enhance the visual appeal and readability of the UI. It requires an API key from Anthropic, indicating that it likely integrates with Anthropic's AI services to process prompts and generate corresponding user interface elements dynamically. The README provides instructions for running the development server using various package managers like npm, yarn, pnpm, bun, or Docker, making it flexible for different development environments. It also includes links to Next.js documentation and tutorials for users who want to learn more about the framework or contribute to the project. Deployment is streamlined through the Vercel platform, which is the recommended way to host Next.js applications, ensuring easy and efficient deployment with minimal configuration. Overall, prompt2ui is a creative and experimental project that leverages modern web technologies and AI integration to transform textual prompts into interactive user interfaces, making it a useful tool for developers and designers interested in exploring AI-driven UI generation.
https://github.com/sullyo/prompt2ui
Also in GUI Action Mapping
mobile-next/mobile-mcp
Mobile Next MCP is a platform-agnostic Model Context Protocol server enabling scalable mobile automation and interaction with iOS and Android devices, simulators, and emulators through accessibility snapshots and coordinate-based controls.
ddupont808/GPT-4V-Act
GPT-4V-Act is an AI agent that uses GPT-4V(ision) to interact with web user interfaces through mouse and keyboard inputs, enabling enhanced accessibility and automation.
naver/egjs
egjs is a modular collection of JavaScript components designed to simplify and accelerate the development of customizable web applications with a focus on ease of use and performance.
njucckevin/SeeClick
SeeClick is a state-of-the-art project providing models, data, and code for advanced visual GUI agents with a focus on GUI grounding, featuring the ScreenSpot benchmark and superior performance on multi-platform GUI element prediction.
OS-Copilot/OS-Atlas
OS-Atlas is a foundation action model for generalist GUI agents that enables precise visual grounding and interaction with UI elements in screenshots using advanced vision-language models.
AriaUI/Aria-UI
Aria-UI is an open-source, fast, and context-aware action grounding system for GUI instructions, achieving state-of-the-art performance in dynamic agent tasks like AndroidWorld and OSWorld.
opendilab/awesome-ui-agents
A curated and continuously updated collection of research papers, models, tools, and datasets focused on UI agents that interact with various user interfaces across platforms like web, mobile, and OS environments.