Scrape parse configs
Generate
Dry run against live pages: the test URLs are fetched through the external scraping service with rendering, then in GENERATE mode an AI model writes the configurations from the human_explained_extractions description and returns them with id 0, while in the other modes the configurations sent in the payload are applied instead.
Nothing is stored, both paths call paid external services, and at most two test URLs are accepted.
POST
/api/{tenant:minlength(2)}/v{version:apiVersion}/scrape_parse_configs/generate
- Authorisation
- Minimum role: ADMIN
Request arguments
Request body example
Response
- Status
- 200 (OK)
- Type
- DTO_scrape_parse_config_generate_output
- Wrapper
- DTO_response_wrap
- Other statuses
- 400 (BadRequest), 403 (Forbidden)
Response body example
| Name | Type | ReadOnly | Description |
|---|
| scrape_domain_id | Int32 | | Scrape domain id |
| configs | DTO_scrape_parse_config[] | | Configs |
| test_urls | String[] | | Test urls |
| mode | ENUM scraper_parse_generate_mode | | Mode |
| human_explained_extractions | String[] | | For generate mode, explain what should be extracted. |
DTO_scrape_parse_config_generate_output
DTO_scrape_parse_config
| Name | Type | ReadOnly | Description |
|---|
| created_by | DTO_reference_user | ReadOnly | Created by |
| last_updated_by | DTO_reference_user | ReadOnly | Last updated by |
| sub_configs | DTO_scrape_parse_config[] | | Sub configs |
| id | Int32 | ReadOnly | Leave empty on input |
| scrape_domain_id | Int32 | | Scrape domain id |
| apply_on_url_pattern | String | | Apply on url pattern |
| unique_key | String | | Unique key |
| xpath | String | | Xpath |
| is_collection | Boolean | | Is collection |
| extraction_mode | ENUM scraper_parse_extraction_mode | | Extraction mode |
| post_process_rule | ENUM scraper_parse_post_process | | Post process rule |
| used_last | DateTime (nullable) | | Will update if it was more than 10 minutes (or similar) to avoid too many updates. |
| created | DateTime | ReadOnly | Created |
| last_updated | DateTime | ReadOnly | Last updated |
DTO_reference_user
| Name | Type | ReadOnly | Description |
|---|
| profile_pic | String | ReadOnly | Profile pic |
| email | String | ReadOnly | Email |
| id | Int32 | | Id |
| label | Object | ReadOnly | Label |
| Name | Type | ReadOnly | Description |
|---|
| test_url | String | | Test url |
| value | String | | Value |
| Value | Name | Description |
|---|
| 0 | NOT_SET | NOT_SET |
| 1 | INNER_HTML | INNER_HTML |
| 2 | INNER_TEXT | INNER_TEXT |
| 3 | ATTRIBUTE_HREF | ATTRIBUTE_HREF |
| 4 | ATTRIBUTE_SRC | ATTRIBUTE_SRC |
| 5 | DATETIME | DATETIME |
| 6 | ARIA_LABEL | ARIA_LABEL |
ENUM scraper_parse_post_process
| Value | Name | Description |
|---|
| 0 | NOT_SET | NOT_SET |
| 1 | PATH_JOIN | PATH_JOIN |
ENUM scraper_parse_generate_mode
| Value | Name | Description |
|---|
| 0 | UNKNOWN | UNKNOWN |
| 1 | TEST | TEST |
| 2 | GENERATE | GENERATE |
| 3 | WARM_UP | WARM_UP |
Generated Aug 4, 2026 18:37
