Skip to documentation
Browse documentation

Universal Web API

Capture, extract, map, crawl, and screenshot public web content.

Platform ID web
Endpoints 6 capabilities
View raw

Capabilities

Every capability uses the same POST /execute envelope. Open an endpoint for its exact ID, input contract, code samples, cost, runtime, and illustrative output.

Archive Page (Compatibility) web.page.archive

Compatibility alias for saved-page capture with automatic inline or hosted delivery up to 50 MB per asset and 500 MB total. Returns explicit outcome and fidelity, requested versus effective asset budgets, per-asset failures, and an embedded upscrape-manifest.json in ZIP artifacts. New integrations should use web.page.capture.

1 credit per request
Capture Page web.page.capture

Capture one public webpage or content URL with fail-closed HTTP, truncation and OCR semantics, physical-attempt evidence, and optional exact-source, single-HTML or ZIP artifacts.

1 credit per request
Extract Page web.page.extract

Capture one public webpage or supported content URL and return data shaped by a caller-provided JSON Schema or fields shorthand. Deterministic extraction uses page metadata, the evidence graph, parsed files, and filtered URL enumeration first; internal AI resolves remaining fields per the ai mode.

1 credit per request
Screenshot Page web.page.screenshot

Capture a host-rendered JPEG screenshot of a public viewport or full page through the typed internal browser acquisition boundary.

1 credit per request
Crawl Site web.site.crawl

Run a bounded breadth-first crawl across same-site pages and public files with robots rules, explicit scope patterns, controlled concurrency, stateless continuation, source-completeness reconciliation, and optional non-destructive AI relevance ranking.

1 credit per request
Map Site web.site.map

Build a deterministic, robots-aware public URL inventory from the starting page, robots.txt, and bounded recursive sitemap indexes. Classifies pages and files without deeply fetching every discovered URL and supports opaque pagination cursors.

1 credit per request

Common uses

  • Turn webpages and public files into traceable structured content, Markdown, and searchable chunks
  • Discover downloadable documents, datasets, media, and resources
  • Map sitemaps and robots-aware site inventories
  • Run bounded, resumable crawls across pages and public files
  • Create JPEG screenshots and durable HTML or ZIP snapshots
  • Extract caller-defined fields with deterministic and optional AI assistance

Integration contract

A 200 response completed inline. A 202 response was queued; poll GET /jobs/:id until it reaches completed or failed.