Universal Web API Capability

Extract Page

Capture one public webpage or supported content URL and return data shaped by a caller-provided JSON Schema or fields shorthand. Deterministic extraction uses page metadata, the evidence graph, parsed files, and filtered URL enumeration first; internal AI resolves remaining fields per the ai mode.

Capability ID web.page.extract

Cost 1 credit per request

Maximum runtime 120 seconds

View raw

# Extract Page

Capture one public webpage or supported content URL and return data shaped by a caller-provided JSON Schema or fields shorthand. Deterministic extraction uses page metadata, the evidence graph, parsed files, and filtered URL enumeration first; internal AI resolves remaining fields per the ai mode.

- Platform: [Universal Web](https://docs.upscrape.com/docs/platforms/web)
- Capability ID: `web.page.extract`
- Cost: 1 credit per request
- Maximum runtime: 120 seconds
- Execute endpoint: `POST https://data.upscrape.com/execute`

## Request

Use the exact public capability ID in the shared execute envelope.

```bash
curl --request POST \
  --url https://data.upscrape.com/execute \
  --header "Authorization: Bearer YOUR_API_KEY" \
  --header "Content-Type: application/json" \
  --header "Prefer: wait=30" \
  --data '{
  "input": {
    "ai": "never",
    "fields": {
      "canonical_url": "canonical url of the page",
      "description": "short page description",
      "title": "page title"
    },
    "url": "https://example.com/"
  },
  "capability": "web.page.extract"
}'
```

## Input

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `accept_statuses` | `array<integer>` | No | Accept statuses supplied for this request. |
| `ai` | `string` | No | Extraction mode. never: deterministic only. auto: deterministic first, internal AI only for unresolved fields (degrades gracefully when AI is unavailable). always: deterministic plus AI. Defaults to auto when fields is used, otherwise never. |
| `ai_enabled` | `boolean` | No | Deprecated alias for ai: always. Prefer the ai parameter. |
| `allow_browser_fallback` | `boolean` | No | Allow browser fallback supplied for this request. |
| `allow_truncated` | `boolean` | No | Allow truncated supplied for this request. |
| `document` | `object` | No | Optional document processing controls used before deterministic or AI field extraction. |
| `document.chunk_chars` | `integer` | No | Chunk chars supplied for this request. |
| `document.continuation_token` | `string` | No | Continuation token supplied for this request. |
| `document.ocr` | `string` | No | Ocr supplied for this request. Allowed values: `auto`, `never`, `always`. |
| `document.outputs` | `array<string>` | No | Requested document outputs. source returns the originally captured bytes: inline through 5.5 MB, otherwise through a hash-verified durable store, and fails if neither path can deliver them. When source is omitted, remote_source is provenance only and contains no bytes. |
| `document.page_end` | `integer` | No | Page end supplied for this request. |
| `document.page_start` | `integer` | No | Page start supplied for this request. |
| `fields` | `object` | No | Shorthand alternative to output_schema: field name mapped to a natural-language description of what to extract. Compiled into a schema internally. Provide exactly one of output_schema or fields. |
| `instructions` | `string` | No | Optional extraction guidance. Do not include secrets. |
| `max_body_bytes` | `integer` | No | Max body bytes supplied for this request. |
| `max_total_bytes` | `integer` | No | Max total bytes supplied for this request. |
| `output_schema` | `object` | No | JSON Schema object describing the desired data shape. Property descriptions double as per-field extraction hints. Constraints such as items.pattern filter deterministic URL enumeration. Provide exactly one of output_schema or fields. |
| `url` | `string` | Yes | Public http(s) webpage or supported content URL to extract from. |

### Example input

```json
{
  "ai": "never",
  "fields": {
    "canonical_url": "canonical url of the page",
    "description": "short page description",
    "title": "page title"
  },
  "url": "https://example.com/"
}
```

## Response

Successful output is returned in `results[0].data`. Raw platform output is intentionally open-ended and may evolve with the upstream source.

### Illustrative sample output

This redacted fixture is an example, not a fixed response schema.

```json
{
  "data": {
    "canonical_url": "https://example.com/",
    "description": null,
    "title": "Example Domain"
  },
  "evidence": [
    {
      "field": "canonical_url",
      "score": 0.92,
      "source": "metadata",
      "text": "https://example.com/",
      "value": "https://example.com/"
    },
    {
      "field": "title",
      "score": 0.9,
      "source": "metadata",
      "text": "Example Domain",
      "value": "Example Domain"
    }
  ],
  "extractor_version": "web-extractor-2026-08-23-universal-v4",
  "fields": {
    "canonical_url": {
      "confidence": 0.92,
      "evidence": "https://example.com/",
      "source": "metadata",
      "value": "https://example.com/"
    },
    "title": {
      "confidence": 0.9,
      "evidence": "Example Domain",
      "source": "metadata",
      "value": "Example Domain"
    }
  },
  "schema_valid": true,
  "semantic_valid": true,
  "semantic_validation_errors": [],
  "stats": {
    "ai_attempts": 0,
    "ai_completion_tokens": 0,
    "ai_enabled": false,
    "ai_mode": "never",
    "ai_prompt_tokens": 0,
    "ai_used": false,
    "capture_backend": "http",
    "deterministic_fields": 2,
    "fetch_ms": 1704,
    "html_bytes": 559,
    "rendered": false,
    "schema_field_count": 3,
    "schema_valid": true,
    "semantic_valid": true,
    "source_blocked": false,
    "text_chars": 142,
    "unresolved_fields": 1,
    "work_units": 1
  },
  "unresolved_fields": [
    "description"
  ],
  "validation_errors": null,
  "warnings": []
}
```
### Illustrative output fields

Derived from the sample above for orientation only. These fields are not a fixed response schema.

| Path | Observed type | Example |
| --- | --- | --- |
| `data` | `object` | 3 fields |
| `data.canonical_url` | `string` | https://example.com/ |
| `data.description` | `null` | null |
| `data.title` | `string` | Example Domain |
| `evidence` | `array` | 2 items |
| `evidence` | `array` | 2 items |
| `extractor_version` | `string` | web-extractor-2026-08-23-universal-v4 |
| `fields` | `object` | 2 fields |
| `fields.canonical_url` | `object` | 4 fields |
| `fields.title` | `object` | 4 fields |
| `schema_valid` | `boolean` | true |
| `semantic_valid` | `boolean` | true |
| `semantic_validation_errors` | `array` | 0 items |
| `stats` | `object` | 18 fields |
| `stats.ai_attempts` | `integer` | 0 |
| `stats.ai_completion_tokens` | `integer` | 0 |
| `stats.ai_enabled` | `boolean` | false |
| `stats.ai_mode` | `string` | never |
| `stats.ai_prompt_tokens` | `integer` | 0 |
| `stats.ai_used` | `boolean` | false |
| `stats.capture_backend` | `string` | http |
| `stats.deterministic_fields` | `integer` | 2 |
| `stats.fetch_ms` | `integer` | 1704 |
| `stats.html_bytes` | `integer` | 559 |
| `stats.rendered` | `boolean` | false |
| `stats.schema_field_count` | `integer` | 3 |
| `stats.schema_valid` | `boolean` | true |
| `stats.semantic_valid` | `boolean` | true |
| `stats.source_blocked` | `boolean` | false |
| `stats.text_chars` | `integer` | 142 |
| `stats.unresolved_fields` | `integer` | 1 |
| `stats.work_units` | `integer` | 1 |
| `unresolved_fields` | `array` | 1 items |
| `unresolved_fields` | `array` | 1 items |
| `validation_errors` | `null` | null |
| `warnings` | `array` | 0 items |

## Execution behavior

A `200` response completed inline. A `202` response was queued; poll `GET /jobs/{id}` until the job reaches `completed` or `failed`. Use an `Idempotency-Key` when retrying must not create a duplicate logical job.

## Related documentation

- [Authentication](https://docs.upscrape.com/docs/api/authentication)
- [Jobs and results](https://docs.upscrape.com/docs/api/jobs)
- [Errors and retries](https://docs.upscrape.com/docs/api/errors)
- [Idempotency](https://docs.upscrape.com/docs/api/idempotency)

POST /execute Shared execution endpoint

Input parameters

accept_statuses array<integer> optional

Accept statuses supplied for this request.

ai string optional

Extraction mode. never: deterministic only. auto: deterministic first, internal AI only for unresolved fields (degrades gracefully when AI is unavailable). always: deterministic plus AI. Defaults to auto when fields is used, otherwise never.

Example never

Available values neverautoalways

ai_enabled boolean optional

Deprecated alias for ai: always. Prefer the ai parameter.

allow_browser_fallback boolean optional

Allow browser fallback supplied for this request.

allow_truncated boolean optional

Allow truncated supplied for this request.

document object optional

Optional document processing controls used before deterministic or AI field extraction.

document.chunk_chars integer optional

Chunk chars supplied for this request.

document.continuation_token string optional

Continuation token supplied for this request.

document.ocr string optional

Ocr supplied for this request. Allowed values: `auto`, `never`, `always`.

Available values autoneveralways

document.outputs array<string> optional

Requested document outputs. source returns the originally captured bytes: inline through 5.5 MB, otherwise through a hash-verified durable store, and fails if neither path can deliver them. When source is omitted, remote_source is provenance only and contains no bytes.

document.page_end integer optional

Page end supplied for this request.

document.page_start integer optional

Page start supplied for this request.

fields object optional

Shorthand alternative to output_schema: field name mapped to a natural-language description of what to extract. Compiled into a schema internally. Provide exactly one of output_schema or fields.

Example {"canonical_url":"canonical url of the page","description":"short page description","title":"page title"}

instructions string optional

Optional extraction guidance. Do not include secrets.

max_body_bytes integer optional

Max body bytes supplied for this request.

max_total_bytes integer optional

Max total bytes supplied for this request.

output_schema object optional

JSON Schema object describing the desired data shape. Property descriptions double as per-field extraction hints. Constraints such as items.pattern filter deterministic URL enumeration. Provide exactly one of output_schema or fields.

url string required

Public http(s) webpage or supported content URL to extract from.

Example https://example.com/

Response

Successful output is returned in results[0].data. Raw platform output is intentionally open-ended and may evolve with the upstream source.

Illustrative sample The full redacted fixture is shown in the response panel.

Illustrative output fields

Derived from the sample above for orientation only. These observed fields are not a fixed response schema.

Path	Observed type	Example
`data`	`object`	3 fields
`data.canonical_url`	`string`	https://example.com/
`data.description`	`null`	null
`data.title`	`string`	Example Domain
`evidence`	`array`	2 items
`evidence`	`array`	2 items
`extractor_version`	`string`	web-extractor-2026-08-23-universal-v4
`fields`	`object`	2 fields
`fields.canonical_url`	`object`	4 fields
`fields.title`	`object`	4 fields
`schema_valid`	`boolean`	true
`semantic_valid`	`boolean`	true
`semantic_validation_errors`	`array`	0 items
`stats`	`object`	18 fields
`stats.ai_attempts`	`integer`	0
`stats.ai_completion_tokens`	`integer`	0
`stats.ai_enabled`	`boolean`	false
`stats.ai_mode`	`string`	never
`stats.ai_prompt_tokens`	`integer`	0
`stats.ai_used`	`boolean`	false
`stats.capture_backend`	`string`	http
`stats.deterministic_fields`	`integer`	2
`stats.fetch_ms`	`integer`	1704
`stats.html_bytes`	`integer`	559
`stats.rendered`	`boolean`	false
`stats.schema_field_count`	`integer`	3
`stats.schema_valid`	`boolean`	true
`stats.semantic_valid`	`boolean`	true
`stats.source_blocked`	`boolean`	false
`stats.text_chars`	`integer`	142
`stats.unresolved_fields`	`integer`	1
`stats.work_units`	`integer`	1
`unresolved_fields`	`array`	1 items
`unresolved_fields`	`array`	1 items
`validation_errors`	`null`	null
`warnings`	`array`	0 items

Execution behavior

A 200 response completed inline. A 202 response was queued; poll GET /jobs/:id until the job reaches completed or failed.