firecrawl/firecrawl
Firecrawl is an advanced web data API that crawls and scrapes entire websites to convert content into clean, LLM-ready markdown or structured data for AI applications.
Awesome AI Agents › Web Automation Systems
EmergenceAI/Agent-E
Agent-E is an advanced agent-based automation system designed to perform tasks on a user's computer, with a current focus on web browser automation. It leverages natural language processing to allow users to interact with web browsers in an intuitive way, enabling automation of various web-based activities. The system is built on the AG2 agent framework and offers a wide range of functionalities such as filling out web forms using personal or external data, searching and sorting products on e-commerce platforms like Amazon, locating specific content on websites including sports scores and contact information, and interacting with web media like YouTube videos by managing playback settings. Additionally, Agent-E can automate tasks on project management tools such as JIRA by filtering issues and streamlining workflows. It also provides personal shopping assistance by suggesting products tailored to user needs. The project supports multi-agent orchestration with enterprise enhancements including advanced logging, role-based access control, and scalable cloud-hosted infrastructure, backed by expert support. Agent-E is designed to be extensible and versatile, encouraging users to create custom tasks beyond the pre-built capabilities. It offers a free trial of its managed web agent and orchestrator, accessible via a web automation API. The project includes detailed setup instructions for various operating systems, environment configuration, and dependency management. It supports integration with large language models (LLMs) like GPT-4 Turbo for enhanced natural language understanding and task execution. The system can be run locally or in the cloud, with options to use Playwright for browser automation or a local Chrome browser profile. It also provides configurable environment variables for model selection, API keys, logging preferences, and additional skill directories. Overall, Agent-E aims to simplify and automate complex web interactions through natural language commands, making it a powerful tool for personal and enterprise automation needs.
https://github.com/EmergenceAI/Agent-E
Also in Web Automation Systems
nanobrowser/nanobrowser
Nanobrowser is an open-source Chrome extension that enables AI-powered web automation through a multi-agent system using user-configured LLM API keys, offering a privacy-focused and cost-effective alternative to commercial tools like OpenAI Operator.
steel-dev/steel-browser
Steel Browser is an open-source browser API that enables developers to build AI-powered web agents and automation tools with full browser control, session management, proxy support, and debugging features, simplifying web automation without infrastructure overhead.
fake-useragent/fake-useragent
fake-useragent is a Python package that provides an up-to-date and customizable user-agent faker using a real-world database for realistic user-agent strings.
ishan0102/vimGPT
vimGPT is a project that combines GPT-4V's vision capabilities with the Vimium keyboard navigation extension to enable AI-assisted web browsing through visual and keyboard interactions.
brightdata/brightdata-mcp
Bright Data MCP is a powerful Model Context Protocol server that enables AI agents and applications to access and extract real-time web data seamlessly, bypassing geo-restrictions and bot protections for enhanced web scraping and navigation.
spider-rs/spider
A concurrency-first web crawler and scraper written in Rust that streams pages as they arrive and renders JavaScript only when needed, with bindings for Node.js and Python and an MCP server for agents.
nottelabs/notte
Notte is an open-source full stack framework that creates intelligent web browsing agents using a perception layer to enable fast, reliable, and cost-effective interactions with websites through large language models.