firecrawl/firecrawl
Firecrawl is an advanced web data API that crawls and scrapes entire websites to convert content into clean, LLM-ready markdown or structured data for AI applications.
Awesome AI Agents › Web Automation Systems
OSU-NLP-Group/SeeAct
SeeAct is a cutting-edge system designed for generalist web agents that autonomously perform tasks on any website, leveraging the power of large multimodal models (LMMs) such as GPT-4V(ision). The project focuses on enabling these agents to interact with live websites through a robust and flexible codebase that supports real-time web browsing and task execution. SeeAct integrates an innovative framework that utilizes LMMs as generalist web agents, allowing them to understand and manipulate web content effectively. The system is built around two main components: a reliable codebase for running web agents on live websites and a framework that harnesses the capabilities of LMMs to act as versatile web agents. The project includes a tool that interfaces with web browsers using Playwright, facilitating communication between the agent and the browser. This tool translates the agent's predicted actions into executable browser events, enabling seamless interaction with web pages. SeeAct supports various models, including OpenAI's GPT-4V, GPT-4 Turbo, and Google's Gemini, among others, making it adaptable to different AI backends. SeeAct also offers features like a crawler mode for automated web navigation and supports multiple grounding strategies to enhance the agent's understanding of web elements. The system is configurable via TOML files, allowing users to customize its behavior and tasks. It is distributed as a Python package, making it accessible for developers to install and integrate into their projects. The project is actively maintained and has been recognized in academic venues such as ICML 2024. It provides extensive documentation, demo videos, and datasets to support research and development in the field of web automation and AI-driven web agents. Overall, SeeAct represents a significant advancement in creating autonomous, intelligent web agents capable of performing complex tasks across diverse websites.
https://github.com/OSU-NLP-Group/SeeAct
Also in Web Automation Systems
nanobrowser/nanobrowser
Nanobrowser is an open-source Chrome extension that enables AI-powered web automation through a multi-agent system using user-configured LLM API keys, offering a privacy-focused and cost-effective alternative to commercial tools like OpenAI Operator.
steel-dev/steel-browser
Steel Browser is an open-source browser API that enables developers to build AI-powered web agents and automation tools with full browser control, session management, proxy support, and debugging features, simplifying web automation without infrastructure overhead.
fake-useragent/fake-useragent
fake-useragent is a Python package that provides an up-to-date and customizable user-agent faker using a real-world database for realistic user-agent strings.
ishan0102/vimGPT
vimGPT is a project that combines GPT-4V's vision capabilities with the Vimium keyboard navigation extension to enable AI-assisted web browsing through visual and keyboard interactions.
brightdata/brightdata-mcp
Bright Data MCP is a powerful Model Context Protocol server that enables AI agents and applications to access and extract real-time web data seamlessly, bypassing geo-restrictions and bot protections for enhanced web scraping and navigation.
spider-rs/spider
A concurrency-first web crawler and scraper written in Rust that streams pages as they arrive and renders JavaScript only when needed, with bindings for Node.js and Python and an MCP server for agents.
nottelabs/notte
Notte is an open-source full stack framework that creates intelligent web browsing agents using a perception layer to enable fast, reliable, and cost-effective interactions with websites through large language models.