Agis Web Scraper

Read any URL as Markdown, scrape by selector, extract structured JSON with AI, snapshot & list links. Built for RAG.

Web & DataMCP + x402from $0.03 Live

Overview

A full web-reading toolkit for agents. Fetch any public page as clean, token-efficient Markdown, or go further: scrape specific elements by CSS selector, list every link, capture a page snapshot (HTML + screenshot + markdown), and run AI-powered structured extraction that turns a URL and a prompt into clean JSON. JavaScript rendering handles SPAs and JS-heavy pages. Ideal for RAG pipelines and agents that need to read, parse or extract from the web. The free reader is available over MCP; the browser-powered actions are pay-per-call x402 HTTP endpoints (USDC on Base).

Connect it

Add the remote MCP server to any client, or call the x402 HTTP endpoint directly.

Claude / Cursor / Windsurf

claude mcp add --transport http web-scraper https://api.agishub.com/mcp/web

mcp.json

{
  "mcpServers": {
    "web-scraper": {
      "url": "https://api.agishub.com/mcp/web"
    }
  }
}

x402 HTTP (pay-per-call)

curl -X POST https://api.agishub.com/paid/extract-structured \
  -H "Content-Type: application/json" \
  -d '{ "url": "https://example.com", "prompt": "the product name and price" }'
# → 402 Payment Required, then pay per call in USDC on Base (x402)

Tools included7

extract	Fetch any public web page and return its main content as clean, token-efficient Markdown (title, description, headings, links, lists). Set render:true to execute JavaScript first for single-page apps or JS-heavy pages that would otherwise come back empty. Built for RAG and for agents that need to read the contents of a URL.
scrape	Extract specific elements from a JavaScript-rendered page by CSS selector. Give a list of selectors (e.g. 'h1', '.price', 'a.product') and get back the text and attributes of every match. Backed by a headless browser, so it works on SPAs and JS-heavy pages.
extract_structured	AI-powered structured extraction: give a URL plus a natural-language prompt and/or a JSON Schema, and get back clean structured JSON (e.g. product name, price, rating). Renders the page in a headless browser first, so it works on SPAs.
snapshot	Capture several representations of a page in one call — rendered HTML plus a PNG screenshot by default, and optionally Markdown and the accessibility tree. Backed by a headless browser. Saves round-trips when an agent needs both the content and a visual of a page.
crawl	Crawl multiple pages of a site, respecting link depth and domain limits. Returns async job_id; results include markdown or HTML per page. Use it to fetch and process many pages of content at once.
map	Discover all URLs reachable from a domain within a link depth limit. Returns a flat list of absolute URLs, respects robots.txt crawl delays. Use it to map a site's structure before crawling.
links	Return every hyperlink on a JavaScript-rendered page as a list of absolute URLs, with options to keep only visible links or only same-site links. Backed by a headless browser. Use it to map a site or seed a crawler.