Extract API
Name the fields you want and get them from any page as JSON.
Tell the Extract API which fields you want, in plain words or as a JSON Schema, and it reads them from any public page. Each value comes back with how sure the extractor is and where on the page it found it.
Call it
curl https://data.upscrape.com/extract \ -H "Authorization: Bearer $UPSCRAPE_API_KEY" \ -H "Content-Type: application/json" \ -H "Prefer: wait=30" \ -d '{ "url": "https://www.allbirds.com/products/mens-tree-runners", "fields": { "name": "product name", "price": "current price", "rating": "average review rating" }, "ai": "never" }'
import osimport requestsjob = requests.post( "https://data.upscrape.com/extract", headers={"Authorization": f"Bearer {os.environ['UPSCRAPE_API_KEY']}", "Prefer": "wait=30"}, json={ "url": "https://www.allbirds.com/products/mens-tree-runners", "fields": {"name": "product name", "price": "current price", "rating": "average review rating"}, "ai": "never", },).json()print(job["results"][0]["data"]["data"])
const res = await fetch("https://data.upscrape.com/extract", { method: "POST", headers: { Authorization: `Bearer ${process.env.UPSCRAPE_API_KEY}`, "Content-Type": "application/json", Prefer: "wait=30", }, body: JSON.stringify({ url: "https://www.allbirds.com/products/mens-tree-runners", fields: { name: "product name", price: "current price", rating: "average review rating" }, ai: "never", }),});const job = await res.json();console.log(job.results[0].data.data);
The fields come back in results[0].data.data. This is from a run on staging on 16 October 2026:
{ "name": "Men's Tree Runners - Black - Everyday Sneakers | Allbirds", "price": "USD 100", "rating": "4.6"}
Say what you want
Use one of three ways:
fields: each field's name and a short description, as above. The quickest start.output_schema: a JSON Schema of the result you want. Use it for types: ask for anumberand you get a number, not "USD 100". Each property'sdescriptiontells the extractor what to look for.mode: a ready-made shape.productreturns a product record;selectorsreads the CSS selectors you give inselectors(add@attrto read an attribute, such asa@href);highlightsreturns the passages that answer yourquestion.
Choose how much AI helps
ai decides what happens when the page doesn't state a field plainly:
ai | What happens |
|---|---|
never | Only what the page states. A field it doesn't hold stays empty and is listed in unresolved_fields. |
auto | The page first; AI fills only the fields left empty. |
always | AI reads every field. |
Every value carries confidence and source in results[0].data.fields, so you can drop answers that aren't sure enough.
Read a list across pages
Set follow_links: true and max_pages (up to 50) to follow the page's own links on the same site and merge the results: lists are joined, and single fields are filled from the first page that has them. Each page visited is part of the one call.
Input
| Field | Type | Required | What it does | Example |
|---|---|---|---|---|
url |
string | yes | The public page or file to read. | https://www.ycombinator.com/companies/stripe |
fields |
object | one of | The fields you want, each with a plain description, such as {"price": "the current price"}. Use this or output_schema. | |
mode |
string | one of | A ready-made shape instead of your own fields: product (a product record), selectors (CSS selectors you give) or highlights (passages that answer a question). | product · selectors · highlights |
output_schema |
object | one of | A JSON Schema of the result you want. Each property's description tells the extractor what to look for. | {…} |
accept_statuses |
array<integer> | no | Error statuses you still want returned, such as [404], to inspect the page. | |
ai |
string | no | never: only what the page states. auto: AI fills the fields the page leaves out. always: AI reads every field. | never · auto · always |
allow_browser_fallback |
boolean | no | Open the page in a real browser when it needs JavaScript or blocks a plain request. On by default. | |
allow_truncated |
boolean | no | Return what was read when a page is larger than the size limit, instead of failing. | |
document |
object | no | Options for PDFs and other files. | |
follow_links |
boolean | no | Also follow the page's own links on the same site and merge what they hold. | |
instructions |
string | no | Extra guidance for the extractor, in plain words. | |
max_body_bytes |
integer | no | The largest page or file to read, in bytes. | |
max_pages |
integer | no | With follow_links, how many pages to visit, the first included. | |
max_total_bytes |
integer | no | The most bytes to read in all, redirects included. | |
question |
string | no | For highlights mode: the question the passages should answer. | |
selectors |
object | no | For selectors mode: each field and the CSS selector to read it from, such as {"price": ".price"}. |
Response
A finished call answers with HTTP 200; the data is in results[0].data, and billing.credits_charged is what it cost. A call still running after the wait answers 202: see jobs and results.
This is results[0].data from a real run:
{ "schema_valid": true, "semantic_valid": true, "unresolved_fields": null, "validation_errors": null, "data": { "batch": "S09", "company": "Stripe", "founded": "2009", "team_size": "7000" }, "fields": { "batch": { "confidence": 0.99, "source": "attribute_json", "value": "S09" }, "company": { "confidence": 0.99, "source": "attribute_json", "value": "Stripe" }, "founded": { "confidence": 0.99, "source": "attribute_json", "value": "2009" }, "team_size": { "confidence": 0.99, "source": "attribute_json", "value": "7000" } }, "semantic_validation_errors": []}
Fields
| Field | Type | What it is | Example |
|---|---|---|---|
data |
object | The fields you asked for, filled in from the page. | 4 fields |
data.company |
string | The company property of the schema in this request: its name. | Stripe |
data.batch |
string | The batch property: the startup's Y Combinator batch. | S09 |
data.founded |
string | The founded property: the year it was founded. | 2009 |
data.team_size |
string | The team_size property: how many people work there, as the page states. | 7000 |
fields |
object | Each field you asked for, with its value, how sure the extraction is and where on the page it was found. | 4 fields |
fields.* |
object | The same field with its value, how sure the extraction is and where it was found. | 3 fields |
fields.*.confidence |
number | How sure the extraction is, from 0 to 1. | 0.99 |
fields.*.source |
string | Where on the page the value was found, such as the page's structured data or its text. | attribute_json |
fields.*.value |
string | The value found for the field. | S09 |
schema_valid |
boolean | Whether the fields came back in the shape you asked for. | true |
semantic_valid |
boolean | Whether the values passed the checks you set. | true |
semantic_validation_errors |
array | Checks the values did not pass, when you set checks in your schema. | empty in this sample |
unresolved_fields |
null | Fields the page did not hold; empty when every field was found. | null |
validation_errors |
null | Where the result did not fit your schema; empty when it fits. | null |
What to know
- Values come back as the page writes them, such as "USD 100". Ask for a number in output_schema to get one.
- With ai set to never, a field the page does not state stays empty and is listed in unresolved_fields; nothing is guessed.
- Each value carries its confidence and source, so you can drop answers that are not sure enough.
- A page that needs JavaScript is opened in a real browser for you; that is still 1 credit.
If a call fails
A failed call costs nothing. The response carries an error.code you can branch on; Errors and retries lists every code and which ones to retry.
Something missing or unclear? Ask the team that builds the endpoints.