Awesome AI AgentsBrowser Automation Tools

oxylabs/ai-crawler-py

⭐ 3263 added to this list on 2025-12-15 repository created 2025-09-17

AI-Crawler by Oxylabs AI Studio is an advanced web crawling and data extraction tool that leverages artificial intelligence to simplify and enhance the process of gathering relevant information from websites. Starting from any given URL, the AI-Crawler intelligently explores the domain, guided by a natural language prompt provided by the user, to identify and extract pertinent pages and data. This approach eliminates the need for traditional, rule-based web scrapers that rely on static selectors and custom scripts, making data acquisition more dynamic and adaptable. The tool supports output in structured JSON or Markdown formats, allowing seamless integration into various automation and AI workflows. For JSON outputs, users can define or automatically generate parsing schemas in natural language, ensuring the extracted data fits specific application requirements. The AI-Crawler also offers advanced features such as AI-assisted URL selection, schema-based parsing, and the ability to handle both static and JavaScript-rendered pages. Additionally, it supports geo-targeting to customize proxy locations. Designed as a low-code solution, AI-Crawler is ideal for developers and data scientists who want to focus on data analysis rather than building and maintaining complex scrapers. It is useful for a wide range of applications including locating terms of service pages, gathering pricing information for market research, extracting company information from “About” pages, and compiling news articles on specific topics. The tool requires an API key and Python 3.10+ for installation and usage, with a free trial offering 1,000 credits. It is part of the Oxylabs AI Studio ecosystem and integrates with their Python SDK. The project provides comprehensive documentation, code examples, and support resources to facilitate easy adoption and effective use.

https://github.com/oxylabs/ai-crawler-py

ai-assisted-url-selectionai-crawlerautomationdata-acquisitiondata-extractiongeo-targetingjavascript-renderingjson-outputlow-code-toolmarkdown-outputnatural-language-promptoxylabs-ai-studiopython-sdkschema-based-parsingweb-crawlingweb-scraper-alternative

Also in Browser Automation Tools

browser-use/browser-use

Browser-use is a project that enables AI agents to control and automate web browsers, allowing users to perform complex tasks through natural language commands and AI-driven browser interactions.

CloakHQ/CloakBrowser

CloakBrowser provides a stealthy Chromium binary with C++ source-level fingerprint patches to pass bot detection, serving as a drop-in Playwright/Puppeteer replacement.

jackwener/OpenCLI

OpenCLI converts any website or local tool into a command-line interface, enabling both human interaction and AI agent automation through logged-in browser sessions.

browserbase/stagehand

Stagehand is a production-ready AI web browsing framework that combines code and natural language to enable reliable and extensible browser automation.

browser-use/web-ui

A web-based user interface, powered by Gradio, for interacting with browser automation agents, supporting various LLMs and custom browser configurations.

citrolabs/ego-lite

ego lite is a specialized browser designed for AI agents to perform web automation in parallel with human browsing, offering isolated workspaces and enhanced page snapshots.

jo-inc/camofox-browser

Camofox-browser is a stealth headless browser server, powered by Camoufox (a Firefox fork), designed for AI agents to bypass bot detection and Cloudflare, offering a Puppeteer/Playwright alternative.

microsoft/fara

Fara-7B is a compact and efficient agentic small language model by Microsoft designed to perform multi-step computer tasks through visual interaction, achieving state-of-the-art performance on web automation benchmarks while enabling on-device deployment for privacy and low latency.