firecrawl/firecrawl
Firecrawl is an advanced web data API that crawls and scrapes entire websites to convert content into clean, LLM-ready markdown or structured data for AI applications.
Awesome AI Agents › Web Automation Systems
idosal/AgentLLM
AgentLLM is a pioneering proof-of-concept project that demonstrates the feasibility of running autonomous agents powered by large language models (LLMs) directly within a web browser. Unlike traditional LLM implementations that rely on powerful remote servers, AgentLLM leverages browser-native LLMs to enable goal-oriented autonomous agents to operate locally on the user's device, enhancing privacy, reducing costs, and improving availability. The project builds upon the WebLLM research, which utilizes the WebGPU API in Chromium-based browsers to perform GPU-accelerated inference, significantly boosting performance compared to CPU-based methods. AgentLLM modifies the existing AgentGPT framework by replacing its backend ChatGPT model with the open-source WizardLM and adapting the prompt mechanism to suit embedded LLMs. This approach allows the autonomous agents to execute complex tasks through iterative task generation and execution loops without relying on external tools, simplifying the environment and improving predictability. The project includes a user-friendly graphical interface for easy interaction and quick prototyping of autonomous agent capabilities. The repository provides multiple setup options, including running a live demo on supported Chromium browsers, local development with Node.js, and containerized deployment using Docker or Docker Compose. It also supports cloud-based development environments like GitHub Codespaces. While the project is experimental and not production-ready, it serves as a valuable research platform to explore the potential of browser-native LLMs in autonomous agent applications. Overall, AgentLLM represents a significant step towards decentralized AI applications by enabling autonomous agents to function efficiently within the browser environment, opening new possibilities for privacy-conscious and accessible AI-driven automation.
https://github.com/idosal/AgentLLM
Also in Web Automation Systems
nanobrowser/nanobrowser
Nanobrowser is an open-source Chrome extension that enables AI-powered web automation through a multi-agent system using user-configured LLM API keys, offering a privacy-focused and cost-effective alternative to commercial tools like OpenAI Operator.
steel-dev/steel-browser
Steel Browser is an open-source browser API that enables developers to build AI-powered web agents and automation tools with full browser control, session management, proxy support, and debugging features, simplifying web automation without infrastructure overhead.
fake-useragent/fake-useragent
fake-useragent is a Python package that provides an up-to-date and customizable user-agent faker using a real-world database for realistic user-agent strings.
ishan0102/vimGPT
vimGPT is a project that combines GPT-4V's vision capabilities with the Vimium keyboard navigation extension to enable AI-assisted web browsing through visual and keyboard interactions.
brightdata/brightdata-mcp
Bright Data MCP is a powerful Model Context Protocol server that enables AI agents and applications to access and extract real-time web data seamlessly, bypassing geo-restrictions and bot protections for enhanced web scraping and navigation.
spider-rs/spider
A concurrency-first web crawler and scraper written in Rust that streams pages as they arrive and renders JavaScript only when needed, with bindings for Node.js and Python and an MCP server for agents.
nottelabs/notte
Notte is an open-source full stack framework that creates intelligent web browsing agents using a perception layer to enable fast, reliable, and cost-effective interactions with websites through large language models.