Universal Web API Capability

Extract page

Capture one public webpage or supported content URL and return data shaped by a caller-provided JSON Schema or fields shorthand. Deterministic extraction uses page metadata, the evidence graph, parsed files, and filtered URL enumeration first; internal AI resolves remaining fields per the ai mode.

Capability ID web.page.extract

Cost From $1.25 per 1,000 requests · 1 credit

Maximum runtime 120 seconds

View raw

# Extract page

Capture one public webpage or supported content URL and return data shaped by a caller-provided JSON Schema or fields shorthand. Deterministic extraction uses page metadata, the evidence graph, parsed files, and filtered URL enumeration first; internal AI resolves remaining fields per the ai mode.

- Platform: [Universal Web](https://docs.upscrape.com/docs/platforms/web)
- Capability ID: `web.page.extract`
- Cost: 1 credit per request
- Maximum runtime: 120 seconds
- Execute endpoint: `POST https://data.upscrape.com/execute`

## Request

Use the exact public capability ID in the shared execute envelope.

```bash
curl --request POST \
  --url https://data.upscrape.com/execute \
  --header "Authorization: Bearer $UPSCRAPE_API_KEY" \
  --header "Content-Type: application/json" \
  --header "Prefer: wait=30" \
  --data '{
  "input": {
    "ai": "never",
    "fields": {
      "heading": "main heading",
      "title": "page title"
    },
    "url": "https://example.com/"
  },
  "capability": "web.page.extract"
}'
```

## Input

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `accept_statuses` | `array<integer>` | No |  |
| `ai` | `string` | No | Extraction mode. never: deterministic only. auto: deterministic first, internal AI only for unresolved fields (degrades gracefully when AI is unavailable). always: deterministic plus AI. Defaults to auto when fields is used, otherwise never. |
| `ai_enabled` | `boolean` | No | Deprecated alias for ai: always. Prefer the ai parameter. |
| `allow_browser_fallback` | `boolean` | No | Default: `false`. |
| `allow_truncated` | `boolean` | No | Default: `false`. |
| `document` | `object` | No | Optional document processing controls used before deterministic or AI field extraction. |
| `document.chunk_chars` | `integer` | No | From 500 to 20000. Default: `4000`. |
| `document.continuation_token` | `string` | No |  |
| `document.ocr` | `string` | No | Allowed values: `auto`, `never`, `always`. Default: `auto`. |
| `document.outputs` | `array<string>` | No | Requested document outputs. source returns the originally captured bytes: inline through 5.5 MB, otherwise through a hash-verified durable store, and fails if neither path can deliver them. When source is omitted, remote_source is provenance only and contains no bytes. |
| `document.page_end` | `integer` | No | From 1 to 1000000. |
| `document.page_start` | `integer` | No | From 1 to 1000000. |
| `fields` | `object` | No | Shorthand alternative to output_schema: field name mapped to a natural-language description of what to extract. Compiled into a schema internally. Provide exactly one of output_schema or fields. |
| `instructions` | `string` | No | Optional extraction guidance. Do not include secrets. |
| `max_body_bytes` | `integer` | No | From 1 to 104857600. |
| `max_total_bytes` | `integer` | No | From 1 to 104857600. |
| `output_schema` | `object` | No | JSON Schema object describing the desired data shape. Property descriptions double as per-field extraction hints. Constraints such as items.pattern filter deterministic URL enumeration. Provide exactly one of output_schema or fields. |
| `url` | `string` | Yes | Public http(s) webpage or supported content URL to extract from. |

### Example input

```json
{
  "ai": "never",
  "fields": {
    "heading": "main heading",
    "title": "page title"
  },
  "url": "https://example.com/"
}
```

## Response

Successful output is returned in `results[0].data`. Raw platform output is intentionally open-ended and may evolve with the upstream source.

### Illustrative sample output

This redacted fixture is an example, not a fixed response schema.

```json
{
  "data": {
    "heading": "Example Domain",
    "title": "Example Domain"
  },
  "evidence": [
    {
      "field": "title",
      "score": 0.9,
      "source": "metadata",
      "text": "Example Domain",
      "value": "Example Domain"
    },
    {
      "field": "heading",
      "score": 0.99,
      "source": "dom_heading",
      "text": "Example Domain",
      "value": "Example Domain"
    }
  ],
  "extractor_version": "web-extractor-2026-08-25-universal-v7",
  "fields": {
    "heading": {
      "confidence": 0.99,
      "evidence": "Example Domain",
      "source": "dom_heading",
      "value": "Example Domain"
    },
    "title": {
      "confidence": 0.9,
      "evidence": "Example Domain",
      "source": "metadata",
      "value": "Example Domain"
    }
  },
  "schema_valid": true,
  "semantic_valid": true,
  "semantic_validation_errors": [],
  "stats": {
    "ai_attempts": 0,
    "ai_completion_tokens": 0,
    "ai_enabled": false,
    "ai_mode": "never",
    "ai_prompt_tokens": 0,
    "ai_used": false,
    "capture_backend": "http",
    "deterministic_fields": 2,
    "fetch_ms": 1704,
    "html_bytes": 559,
    "rendered": false,
    "schema_field_count": 2,
    "schema_valid": true,
    "semantic_valid": true,
    "source_blocked": false,
    "text_chars": 142,
    "unresolved_fields": 0,
    "work_units": 1
  },
  "unresolved_fields": [],
  "validation_errors": null,
  "warnings": []
}
```
### Illustrative output fields

Derived from the sample above for orientation only. These fields are not a fixed response schema.

| Path | Observed type | Example |
| --- | --- | --- |
| `data` | `object` | 2 fields |
| `data.heading` | `string` | Example Domain |
| `data.title` | `string` | Example Domain |
| `evidence` | `array` | 2 items |
| `evidence` | `array` | 2 items |
| `extractor_version` | `string` | web-extractor-2026-08-25-universal-v7 |
| `fields` | `object` | 2 fields |
| `fields.heading` | `object` | 4 fields |
| `fields.title` | `object` | 4 fields |
| `schema_valid` | `boolean` | true |
| `semantic_valid` | `boolean` | true |
| `semantic_validation_errors` | `array` | 0 items |
| `stats` | `object` | 18 fields |
| `stats.ai_attempts` | `integer` | 0 |
| `stats.ai_completion_tokens` | `integer` | 0 |
| `stats.ai_enabled` | `boolean` | false |
| `stats.ai_mode` | `string` | never |
| `stats.ai_prompt_tokens` | `integer` | 0 |
| `stats.ai_used` | `boolean` | false |
| `stats.capture_backend` | `string` | http |
| `stats.deterministic_fields` | `integer` | 2 |
| `stats.fetch_ms` | `integer` | 1704 |
| `stats.html_bytes` | `integer` | 559 |
| `stats.rendered` | `boolean` | false |
| `stats.schema_field_count` | `integer` | 2 |
| `stats.schema_valid` | `boolean` | true |
| `stats.semantic_valid` | `boolean` | true |
| `stats.source_blocked` | `boolean` | false |
| `stats.text_chars` | `integer` | 142 |
| `stats.unresolved_fields` | `integer` | 0 |
| `stats.work_units` | `integer` | 1 |
| `unresolved_fields` | `array` | 0 items |
| `validation_errors` | `null` | null |
| `warnings` | `array` | 0 items |

## Execution behavior

A `200` response completed inline. A `202` response was queued; poll `GET /jobs/{id}` until the job reaches `completed` or `failed`. Use an `Idempotency-Key` when retrying must not create a duplicate logical job.

## Related documentation

- [Authentication](https://docs.upscrape.com/docs/api/authentication)
- [Jobs and results](https://docs.upscrape.com/docs/api/jobs)
- [Errors and retries](https://docs.upscrape.com/docs/api/errors)
- [Idempotency](https://docs.upscrape.com/docs/api/idempotency)

POST /execute Shared execution endpoint

Input parameters

accept_statuses array<integer> optional

ai string optional

Extraction mode. never: deterministic only. auto: deterministic first, internal AI only for unresolved fields (degrades gracefully when AI is unavailable). always: deterministic plus AI. Defaults to auto when fields is used, otherwise never.

Example never

Available values neverautoalways

ai_enabled boolean optional

Deprecated alias for ai: always. Prefer the ai parameter.

allow_browser_fallback boolean optional

Default: `false`.

allow_truncated boolean optional

Default: `false`.

document object optional

Optional document processing controls used before deterministic or AI field extraction.

document.chunk_chars integer optional

From 500 to 20000. Default: `4000`.

document.continuation_token string optional

document.ocr string optional

Allowed values: `auto`, `never`, `always`. Default: `auto`.

Available values autoneveralways

document.outputs array<string> optional

Requested document outputs. source returns the originally captured bytes: inline through 5.5 MB, otherwise through a hash-verified durable store, and fails if neither path can deliver them. When source is omitted, remote_source is provenance only and contains no bytes.

document.page_end integer optional

From 1 to 1000000.

document.page_start integer optional

From 1 to 1000000.

fields object optional

Shorthand alternative to output_schema: field name mapped to a natural-language description of what to extract. Compiled into a schema internally. Provide exactly one of output_schema or fields.

Example {"heading":"main heading","title":"page title"}

instructions string optional

Optional extraction guidance. Do not include secrets.

max_body_bytes integer optional

From 1 to 104857600.

max_total_bytes integer optional

From 1 to 104857600.

output_schema object optional

JSON Schema object describing the desired data shape. Property descriptions double as per-field extraction hints. Constraints such as items.pattern filter deterministic URL enumeration. Provide exactly one of output_schema or fields.

url string required

Public http(s) webpage or supported content URL to extract from.

Example https://example.com/

Response

Successful output is returned in results[0].data. Raw platform output is intentionally open-ended and may evolve with the upstream source.

Illustrative sample The full redacted fixture is shown in the response panel.

Illustrative output fields

Derived from the sample above for orientation only. These observed fields are not a fixed response schema.

Path	Observed type	Example
`data`	`object`	2 fields
`data.heading`	`string`	Example Domain
`data.title`	`string`	Example Domain
`evidence`	`array`	2 items
`evidence`	`array`	2 items
`extractor_version`	`string`	web-extractor-2026-08-25-universal-v7
`fields`	`object`	2 fields
`fields.heading`	`object`	4 fields
`fields.title`	`object`	4 fields
`schema_valid`	`boolean`	true
`semantic_valid`	`boolean`	true
`semantic_validation_errors`	`array`	0 items
`stats`	`object`	18 fields
`stats.ai_attempts`	`integer`	0
`stats.ai_completion_tokens`	`integer`	0
`stats.ai_enabled`	`boolean`	false
`stats.ai_mode`	`string`	never
`stats.ai_prompt_tokens`	`integer`	0
`stats.ai_used`	`boolean`	false
`stats.capture_backend`	`string`	http
`stats.deterministic_fields`	`integer`	2
`stats.fetch_ms`	`integer`	1704
`stats.html_bytes`	`integer`	559
`stats.rendered`	`boolean`	false
`stats.schema_field_count`	`integer`	2
`stats.schema_valid`	`boolean`	true
`stats.semantic_valid`	`boolean`	true
`stats.source_blocked`	`boolean`	false
`stats.text_chars`	`integer`	142
`stats.unresolved_fields`	`integer`	0
`stats.work_units`	`integer`	1
`unresolved_fields`	`array`	0 items
`validation_errors`	`null`	null
`warnings`	`array`	0 items

Execution behavior

A 200 response completed inline. A 202 response was queued; poll GET /jobs/:id until the job reaches completed or failed.