Scrape domains / List items
Lists the scraped pages of a domain, paginated. Optionally filtered by status and a URL search.
Route: /api/{tenant:minlength(2)}/v{version:apiVersion}/scrape_domains/{id}/items
Method: GET
Authorisation: Minimum role: USER
Request Arguments
| Name | Type | Source |
|---|---|---|
| id | Int32 | |
| take | Int32 | Query |
| page | Int32 | Query |
| status | ENUM scraped_item_status | Query |
| search | String | Query |
| sort_by_key | String | Query |
| sort_order | String | Query |
Request body example
Response object
Response status: 200 (OK)
Response type: DTO_scraped_items_page
Wrapper: DTO_response_wrap
Other response statuses: 400 (BadRequest), 403 (Forbidden)
Response body example
DTO_scraped_items_page
| Name | Type | ReadOnly | Description |
|---|---|---|---|
| items | DTO_scraped_item[] | Items | |
| total_count | Int64 | Total matching items (for pagination) |
DTO_scraped_item
| Name | Type | ReadOnly | Description |
|---|---|---|---|
| id | Int32 | ReadOnly | Id |
| url | String | ReadOnly | Url |
| title | String | ReadOnly | Title |
| status | ENUM scraped_item_status | ReadOnly | Status |
| language | String | ReadOnly | Detected content language (html lang / hreflang) |
| duplicate_of_url | String | ReadOnly | When status is IGNORED_DUPLICATE, the URL of the page that was kept instead |
| created | DateTime | ReadOnly | Created |
| last_crawled | DateTime (nullable) | ReadOnly | Last crawled |
| last_crawled_status_code | Int32 (nullable) | ReadOnly | Last crawled status code |
ENUM scraped_item_status
| Value | Name | Description |
|---|---|---|
| 0 | UNKNOWN | UNKNOWN |
| 1 | TO_BE_CRAWLED | TO_BE_CRAWLED |
| 2 | QUEUED | QUEUED |
| 3 | SCRAPED_SUCCESS | SCRAPED_SUCCESS |
| 4 | SCRAPED_ERROR | SCRAPED_ERROR |
| 5 | IGNORED_DUPLICATE | IGNORED_DUPLICATE |
| 6 | IGNORED_FILTERED | IGNORED_FILTERED |
