Awesome AI Agents › Web Agents
AI-Scraper is an experimental web extraction tool from Oxylabs AI Studio, distributed to Python users through the oxylabs-ai-studio package. Instead of maintaining CSS or XPath selectors and custom parsers, the caller supplies a target URL and a plain-English description of the data to collect, and the hosted scrape agent identifies and parses the relevant fields on the page. Output can be returned as structured JSON for automation pipelines and APIs, or as Markdown when a readable document is wanted, which is the format the vendor suggests for feeding downstream language model workflows. Schemas can be generated automatically from the prompt through a generate_schema call, or supplied by hand as an OpenAPI schema when the exact JSON structure matters. Request parameters cover the target URL, the output format, the schema, a flag for JavaScript rendering of client-side pages, and a geo_location value in ISO2 form that selects the proxy region used for the fetch. Usage requires an AI Studio API key; the vendor offers a free trial with a limited credit allowance, and Python 3.10 or newer is required. The README documents typical applications: pulling product names, descriptions and prices from e-commerce listings, parsing news article titles, dates, authors and body text, collecting structured pricing pages for market research, and capturing job postings with titles, locations, salaries and dates. A parallel JavaScript SDK exists for Node users, and the same AI Studio platform exposes related crawling and search applications. The repository itself is small: it is primarily documentation and code examples pointing at the published SDK and the hosted service rather than a self-contained scraping engine, so the extraction logic runs on the Oxylabs side and is not open source.
https://github.com/oxylabs/ai-scraper-py
LaVague is an open-source Large Action Model framework for developing AI Web Agents that automate web-based tasks using components like a World Model and Action Engine with support for Selenium, Playwright, and Chrome extension drivers.
Index is an advanced open-source browser agent that autonomously performs complex web tasks using AI models, offering hosted and self-hosted options with strong observability features.
Tarsier is a vision utility that enhances web interaction agents by tagging webpage elements and converting screenshots into text representations to enable effective LLM-based web automation.
A curated list of tools, frameworks, and resources for building AI web agents that autonomously browse and interact with the web.
WebVoyager is an end-to-end web agent powered by Large Multimodal Models that interacts with real-world websites to complete diverse user tasks using textual and visual information.
AgentLab is an open-source framework for developing, testing, and benchmarking scalable and reproducible web agents across diverse tasks using BrowserGym benchmarks and large-scale parallel experiments.
WebShop is a simulated e-commerce environment designed to train and evaluate grounded language agents for complex real-world web interaction tasks involving product search and purchase based on natural language instructions.
Ash Agents is a Python swarm framework that performs web tasks through disposable agents: each runs in an isolated browser context with its own fingerprint and egress, then is destroyed so no interaction links back to the user.