# Review ## Create Extract Review `$ llp extract:review create` **post** `/api/v2/extract/{job_id}/review` Beta: answers 403 until extract review is enabled for the caller and their organization. Turn review on for an extract job. Fields read below `confidence_threshold`, and required fields left empty, are flagged for a person to check. Every setting has a default, so the body can be left out. A job still running can be named: it waits for review once it completes. A job that failed or was cancelled without a review, or that cannot be reviewed, answers 409 with `code: job_not_reviewable`; a field in `fields_in_scope` the job's schema does not have answers 400. Answers 201 with the new review, or 200 with the job's existing review, whose settings stay as they are. ### Parameters - `--job-id: string` Path param - `--organization-id: optional string` Query param - `--project-id: optional string` Query param - `--confidence-threshold: optional number` Body param: Fields read below this confidence are flagged. - `--fields-in-scope: optional array of string` Body param: JSON Pointers of the fields the threshold applies to; empty means all. - `--required-field: optional array of string` Body param: JSON Pointers of fields flagged when empty; defaults to the schema's required. ### Returns - `extract_review: object { id, confidence_threshold, created_at, 16 more }` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: string` Unique identifier of the review. - `confidence_threshold: number` Fields read below this confidence are flagged. - `created_at: string` When review was turned on. - `decided_at: string` When it was approved or rejected. - `decided_by: string` ID of the user who approved or rejected it. - `decided_reason: string` Why it was rejected. - `field_actions: map[object { action, actor_id, at, value } ]` The latest action per field, keyed by JSON Pointer. - `action: "clear" or "confirm" or "correct"` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: string` ID of the user who acted. - `at: string` When they acted. - `value: optional unknown` The corrected value, for `correct`. - `fields_in_scope: array of string` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: array of object { handled, path, reason, 2 more }` Fields the review asks a person to check. - `handled: boolean` Whether someone confirmed, corrected or cleared it. - `path: string` JSON Pointer of the field. - `reason: "below_threshold" or "required_empty"` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: optional object { row, column }` Where in a list the least confident value is. - `row: number` Index of the row, from 0. - `column: optional string` Property of the row, when rows are objects. - `confidence: optional number` The field's confidence; for a list, its least confident value. - `job_id: string` The extract job reviewed. - `open_flags: array of string` JSON Pointers of flagged fields nobody checked. - `original_result: map[unknown]` The result as the model read it. - `required_fields: array of string` JSON Pointers of fields flagged when empty. - `retain_until: string` When the job and its document may be deleted. - `reviewed_result: map[unknown]` The result with the field actions. - `revision: number` Bumped by every change; actions name the one they saw. - `source_file_id: string` The file the job read, or the file behind the parse job it read; null for a URL. - `status: "approved" or "pending" or "rejected"` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: string` When the review last changed. ### Example ```cli llp extract:review create \ --api-key 'My API Key' \ --job-id job_id ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## Get Extract Review `$ llp extract:review get` **get** `/api/v2/extract/{job_id}/review` Beta: answers 403 until extract review is enabled for the caller and their organization. Get an extract job's review: its flags, field actions and decision, with the result as the model read it and as reviewed. A job without a review answers 404. ### Parameters - `--job-id: string` - `--organization-id: optional string` - `--project-id: optional string` ### Returns - `extract_review: object { id, confidence_threshold, created_at, 16 more }` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: string` Unique identifier of the review. - `confidence_threshold: number` Fields read below this confidence are flagged. - `created_at: string` When review was turned on. - `decided_at: string` When it was approved or rejected. - `decided_by: string` ID of the user who approved or rejected it. - `decided_reason: string` Why it was rejected. - `field_actions: map[object { action, actor_id, at, value } ]` The latest action per field, keyed by JSON Pointer. - `action: "clear" or "confirm" or "correct"` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: string` ID of the user who acted. - `at: string` When they acted. - `value: optional unknown` The corrected value, for `correct`. - `fields_in_scope: array of string` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: array of object { handled, path, reason, 2 more }` Fields the review asks a person to check. - `handled: boolean` Whether someone confirmed, corrected or cleared it. - `path: string` JSON Pointer of the field. - `reason: "below_threshold" or "required_empty"` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: optional object { row, column }` Where in a list the least confident value is. - `row: number` Index of the row, from 0. - `column: optional string` Property of the row, when rows are objects. - `confidence: optional number` The field's confidence; for a list, its least confident value. - `job_id: string` The extract job reviewed. - `open_flags: array of string` JSON Pointers of flagged fields nobody checked. - `original_result: map[unknown]` The result as the model read it. - `required_fields: array of string` JSON Pointers of fields flagged when empty. - `retain_until: string` When the job and its document may be deleted. - `reviewed_result: map[unknown]` The result with the field actions. - `revision: number` Bumped by every change; actions name the one they saw. - `source_file_id: string` The file the job read, or the file behind the parse job it read; null for a URL. - `status: "approved" or "pending" or "rejected"` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: string` When the review last changed. ### Example ```cli llp extract:review get \ --api-key 'My API Key' \ --job-id job_id ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## Change Extract Review Fields `$ llp extract:review change-fields` **post** `/api/v2/extract/{job_id}/review/fields` Beta: answers 403 until extract review is enabled for the caller and their organization. Confirm, correct, clear, restore or undo fields of a completed job. The changes apply in order, in one write. A job without a review gets one with the default settings, at revision 0. A path that is not a field of the job, or a value the field's schema refuses, answers 400 and changes nothing. A job that has not completed answers 409 with `code: job_not_reviewable`. A `revision` other than the review's latest answers 409 with `code: stale_revision` and the latest `revision`. ### Parameters - `--job-id: string` Path param - `--change: array of object { change, path, value }` Body param: Changes to apply, in order. - `--revision: number` Body param: The review's revision as the caller last read it. - `--organization-id: optional string` Query param - `--project-id: optional string` Query param ### Returns - `extract_review: object { id, confidence_threshold, created_at, 16 more }` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: string` Unique identifier of the review. - `confidence_threshold: number` Fields read below this confidence are flagged. - `created_at: string` When review was turned on. - `decided_at: string` When it was approved or rejected. - `decided_by: string` ID of the user who approved or rejected it. - `decided_reason: string` Why it was rejected. - `field_actions: map[object { action, actor_id, at, value } ]` The latest action per field, keyed by JSON Pointer. - `action: "clear" or "confirm" or "correct"` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: string` ID of the user who acted. - `at: string` When they acted. - `value: optional unknown` The corrected value, for `correct`. - `fields_in_scope: array of string` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: array of object { handled, path, reason, 2 more }` Fields the review asks a person to check. - `handled: boolean` Whether someone confirmed, corrected or cleared it. - `path: string` JSON Pointer of the field. - `reason: "below_threshold" or "required_empty"` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: optional object { row, column }` Where in a list the least confident value is. - `row: number` Index of the row, from 0. - `column: optional string` Property of the row, when rows are objects. - `confidence: optional number` The field's confidence; for a list, its least confident value. - `job_id: string` The extract job reviewed. - `open_flags: array of string` JSON Pointers of flagged fields nobody checked. - `original_result: map[unknown]` The result as the model read it. - `required_fields: array of string` JSON Pointers of fields flagged when empty. - `retain_until: string` When the job and its document may be deleted. - `reviewed_result: map[unknown]` The result with the field actions. - `revision: number` Bumped by every change; actions name the one they saw. - `source_file_id: string` The file the job read, or the file behind the parse job it read; null for a URL. - `status: "approved" or "pending" or "rejected"` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: string` When the review last changed. ### Example ```cli llp extract:review change-fields \ --api-key 'My API Key' \ --job-id job_id \ --change '{change: correct, path: /total}' \ --revision 0 ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## Approve Extract Review `$ llp extract:review approve` **post** `/api/v2/extract/{job_id}/review/approve` Beta: answers 403 until extract review is enabled for the caller and their organization. Approve the reviewed result. While flagged fields nobody checked remain, answers 409 with `code: open_flags` and their `paths`, unless `sign_off` confirms them as part of approving. A job that has not completed answers 409 with `code: job_not_reviewable`. A `revision` other than the review's latest answers 409 with `code: stale_revision` and the latest `revision`. ### Parameters - `--job-id: string` Path param - `--revision: number` Body param: The review's revision as the caller last read it. - `--organization-id: optional string` Query param - `--project-id: optional string` Query param - `--sign-off: optional boolean` Body param: Confirm the flagged fields nobody checked as part of approving. ### Returns - `extract_review: object { id, confidence_threshold, created_at, 16 more }` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: string` Unique identifier of the review. - `confidence_threshold: number` Fields read below this confidence are flagged. - `created_at: string` When review was turned on. - `decided_at: string` When it was approved or rejected. - `decided_by: string` ID of the user who approved or rejected it. - `decided_reason: string` Why it was rejected. - `field_actions: map[object { action, actor_id, at, value } ]` The latest action per field, keyed by JSON Pointer. - `action: "clear" or "confirm" or "correct"` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: string` ID of the user who acted. - `at: string` When they acted. - `value: optional unknown` The corrected value, for `correct`. - `fields_in_scope: array of string` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: array of object { handled, path, reason, 2 more }` Fields the review asks a person to check. - `handled: boolean` Whether someone confirmed, corrected or cleared it. - `path: string` JSON Pointer of the field. - `reason: "below_threshold" or "required_empty"` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: optional object { row, column }` Where in a list the least confident value is. - `row: number` Index of the row, from 0. - `column: optional string` Property of the row, when rows are objects. - `confidence: optional number` The field's confidence; for a list, its least confident value. - `job_id: string` The extract job reviewed. - `open_flags: array of string` JSON Pointers of flagged fields nobody checked. - `original_result: map[unknown]` The result as the model read it. - `required_fields: array of string` JSON Pointers of fields flagged when empty. - `retain_until: string` When the job and its document may be deleted. - `reviewed_result: map[unknown]` The result with the field actions. - `revision: number` Bumped by every change; actions name the one they saw. - `source_file_id: string` The file the job read, or the file behind the parse job it read; null for a URL. - `status: "approved" or "pending" or "rejected"` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: string` When the review last changed. ### Example ```cli llp extract:review approve \ --api-key 'My API Key' \ --job-id job_id \ --revision 0 ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## Reject Extract Review `$ llp extract:review reject` **post** `/api/v2/extract/{job_id}/review/reject` Beta: answers 403 until extract review is enabled for the caller and their organization. Reject the result, with a reason. A job that failed or was cancelled can be rejected, which closes a review turned on while it ran. A job still queued or running answers 409 with `code: job_not_reviewable`. A `revision` other than the review's latest answers 409 with `code: stale_revision` and the latest `revision`. ### Parameters - `--job-id: string` Path param - `--reason: string` Body param: Why the result was rejected. - `--revision: number` Body param: The review's revision as the caller last read it. - `--organization-id: optional string` Query param - `--project-id: optional string` Query param ### Returns - `extract_review: object { id, confidence_threshold, created_at, 16 more }` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: string` Unique identifier of the review. - `confidence_threshold: number` Fields read below this confidence are flagged. - `created_at: string` When review was turned on. - `decided_at: string` When it was approved or rejected. - `decided_by: string` ID of the user who approved or rejected it. - `decided_reason: string` Why it was rejected. - `field_actions: map[object { action, actor_id, at, value } ]` The latest action per field, keyed by JSON Pointer. - `action: "clear" or "confirm" or "correct"` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: string` ID of the user who acted. - `at: string` When they acted. - `value: optional unknown` The corrected value, for `correct`. - `fields_in_scope: array of string` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: array of object { handled, path, reason, 2 more }` Fields the review asks a person to check. - `handled: boolean` Whether someone confirmed, corrected or cleared it. - `path: string` JSON Pointer of the field. - `reason: "below_threshold" or "required_empty"` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: optional object { row, column }` Where in a list the least confident value is. - `row: number` Index of the row, from 0. - `column: optional string` Property of the row, when rows are objects. - `confidence: optional number` The field's confidence; for a list, its least confident value. - `job_id: string` The extract job reviewed. - `open_flags: array of string` JSON Pointers of flagged fields nobody checked. - `original_result: map[unknown]` The result as the model read it. - `required_fields: array of string` JSON Pointers of fields flagged when empty. - `retain_until: string` When the job and its document may be deleted. - `reviewed_result: map[unknown]` The result with the field actions. - `revision: number` Bumped by every change; actions name the one they saw. - `source_file_id: string` The file the job read, or the file behind the parse job it read; null for a URL. - `status: "approved" or "pending" or "rejected"` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: string` When the review last changed. ### Example ```cli llp extract:review reject \ --api-key 'My API Key' \ --job-id job_id \ --reason 'The invoice total does not match the line items.' \ --revision 0 ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## Reopen Extract Review `$ llp extract:review reopen` **post** `/api/v2/extract/{job_id}/review/reopen` Beta: answers 403 until extract review is enabled for the caller and their organization. Put an approved or rejected review back to pending. Its field actions stay. A `revision` other than the review's latest answers 409 with `code: stale_revision` and the latest `revision`. ### Parameters - `--job-id: string` Path param - `--revision: number` Body param: The review's revision as the caller last read it. - `--organization-id: optional string` Query param - `--project-id: optional string` Query param ### Returns - `extract_review: object { id, confidence_threshold, created_at, 16 more }` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: string` Unique identifier of the review. - `confidence_threshold: number` Fields read below this confidence are flagged. - `created_at: string` When review was turned on. - `decided_at: string` When it was approved or rejected. - `decided_by: string` ID of the user who approved or rejected it. - `decided_reason: string` Why it was rejected. - `field_actions: map[object { action, actor_id, at, value } ]` The latest action per field, keyed by JSON Pointer. - `action: "clear" or "confirm" or "correct"` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: string` ID of the user who acted. - `at: string` When they acted. - `value: optional unknown` The corrected value, for `correct`. - `fields_in_scope: array of string` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: array of object { handled, path, reason, 2 more }` Fields the review asks a person to check. - `handled: boolean` Whether someone confirmed, corrected or cleared it. - `path: string` JSON Pointer of the field. - `reason: "below_threshold" or "required_empty"` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: optional object { row, column }` Where in a list the least confident value is. - `row: number` Index of the row, from 0. - `column: optional string` Property of the row, when rows are objects. - `confidence: optional number` The field's confidence; for a list, its least confident value. - `job_id: string` The extract job reviewed. - `open_flags: array of string` JSON Pointers of flagged fields nobody checked. - `original_result: map[unknown]` The result as the model read it. - `required_fields: array of string` JSON Pointers of fields flagged when empty. - `retain_until: string` When the job and its document may be deleted. - `reviewed_result: map[unknown]` The result with the field actions. - `revision: number` Bumped by every change; actions name the one they saw. - `source_file_id: string` The file the job read, or the file behind the parse job it read; null for a URL. - `status: "approved" or "pending" or "rejected"` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: string` When the review last changed. ### Example ```cli llp extract:review reopen \ --api-key 'My API Key' \ --job-id job_id \ --revision 0 ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## List Extract Review Events `$ llp extract:review list-events` **get** `/api/v2/extract/{job_id}/review/events` Beta: answers 403 until extract review is enabled for the caller and their organization. List a review's history, newest first. ### Parameters - `--job-id: string` - `--organization-id: optional string` - `--page-size: optional number` Events per page. - `--page-token: optional string` Token from the previous page. - `--project-id: optional string` ### Returns - `ExtractReviewEventListResponse: object { items, next_page_token, total_size }` A page of a review's history, newest first. - `items: array of ExtractReviewEvent` The list of items. - `id: string` Unique identifier of the entry. - `actor_id: string` ID of the user who acted. - `created_at: string` When it happened. - `kind: "approved" or "created" or "field_cleared" or 7 more` What happened. - `"approved"` - `"created"` - `"field_cleared"` - `"field_confirmed"` - `"field_corrected"` - `"field_restored"` - `"field_undone"` - `"flags_signed_off"` - `"rejected"` - `"reopened"` - `detail: optional object { confidence_threshold, fields_in_scope, origin, required_fields } or object { reason } or object { paths }` The settings for `created`, the reason for `rejected`, and the fields for `flags_signed_off`; null for the rest. - `ExtractReviewCreatedDetail: object { confidence_threshold, fields_in_scope, origin, required_fields }` What a `created` entry records: where review was turned on, and its settings. - `confidence_threshold: number` Fields read below this confidence are flagged. - `fields_in_scope: array of string` JSON Pointers of the fields the threshold applies to; empty means all. - `origin: "api" or "edit" or "history" or "run"` Turned on as the job ran, from History, by a first edit, or by the API. - `"api"` - `"edit"` - `"history"` - `"run"` - `required_fields: array of string` JSON Pointers of fields flagged when empty. - `ExtractReviewRejectedDetail: object { reason }` What a `rejected` entry records. - `reason: string` Why the result was rejected. - `ExtractReviewSignedOffDetail: object { paths }` What a `flags_signed_off` entry records. - `paths: array of string` JSON Pointers of the flagged fields confirmed as part of approving. - `path: optional string` JSON Pointer of the field it changed. - `value_after: optional unknown` The field's value after the change. - `value_before: optional unknown` The field's value before the change. - `next_page_token: optional string` A token, which can be sent as page_token to retrieve the next page. If this field is omitted, there are no subsequent pages. - `total_size: optional number` The total number of items available. This is only populated when specifically requested. The value may be an estimate and can be used for display purposes only. ### Example ```cli llp extract:review list-events \ --api-key 'My API Key' \ --job-id job_id ``` #### Response ```json { "items": [ { "id": "id", "actor_id": "actor_id", "created_at": "2019-12-27T18:11:19.117Z", "kind": "approved", "detail": { "confidence_threshold": 0, "fields_in_scope": [ "string" ], "origin": "api", "required_fields": [ "string" ] }, "path": "path", "value_after": {}, "value_before": {} } ], "next_page_token": "next_page_token", "total_size": 0 } ``` ## Domain Types ### Extract Review - `extract_review: object { id, confidence_threshold, created_at, 16 more }` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: string` Unique identifier of the review. - `confidence_threshold: number` Fields read below this confidence are flagged. - `created_at: string` When review was turned on. - `decided_at: string` When it was approved or rejected. - `decided_by: string` ID of the user who approved or rejected it. - `decided_reason: string` Why it was rejected. - `field_actions: map[object { action, actor_id, at, value } ]` The latest action per field, keyed by JSON Pointer. - `action: "clear" or "confirm" or "correct"` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: string` ID of the user who acted. - `at: string` When they acted. - `value: optional unknown` The corrected value, for `correct`. - `fields_in_scope: array of string` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: array of object { handled, path, reason, 2 more }` Fields the review asks a person to check. - `handled: boolean` Whether someone confirmed, corrected or cleared it. - `path: string` JSON Pointer of the field. - `reason: "below_threshold" or "required_empty"` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: optional object { row, column }` Where in a list the least confident value is. - `row: number` Index of the row, from 0. - `column: optional string` Property of the row, when rows are objects. - `confidence: optional number` The field's confidence; for a list, its least confident value. - `job_id: string` The extract job reviewed. - `open_flags: array of string` JSON Pointers of flagged fields nobody checked. - `original_result: map[unknown]` The result as the model read it. - `required_fields: array of string` JSON Pointers of fields flagged when empty. - `retain_until: string` When the job and its document may be deleted. - `reviewed_result: map[unknown]` The result with the field actions. - `revision: number` Bumped by every change; actions name the one they saw. - `source_file_id: string` The file the job read, or the file behind the parse job it read; null for a URL. - `status: "approved" or "pending" or "rejected"` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: string` When the review last changed. ### Extract Review Event - `extract_review_event: object { id, actor_id, created_at, 5 more }` One entry in a review's history. - `id: string` Unique identifier of the entry. - `actor_id: string` ID of the user who acted. - `created_at: string` When it happened. - `kind: "approved" or "created" or "field_cleared" or 7 more` What happened. - `"approved"` - `"created"` - `"field_cleared"` - `"field_confirmed"` - `"field_corrected"` - `"field_restored"` - `"field_undone"` - `"flags_signed_off"` - `"rejected"` - `"reopened"` - `detail: optional object { confidence_threshold, fields_in_scope, origin, required_fields } or object { reason } or object { paths }` The settings for `created`, the reason for `rejected`, and the fields for `flags_signed_off`; null for the rest. - `ExtractReviewCreatedDetail: object { confidence_threshold, fields_in_scope, origin, required_fields }` What a `created` entry records: where review was turned on, and its settings. - `confidence_threshold: number` Fields read below this confidence are flagged. - `fields_in_scope: array of string` JSON Pointers of the fields the threshold applies to; empty means all. - `origin: "api" or "edit" or "history" or "run"` Turned on as the job ran, from History, by a first edit, or by the API. - `"api"` - `"edit"` - `"history"` - `"run"` - `required_fields: array of string` JSON Pointers of fields flagged when empty. - `ExtractReviewRejectedDetail: object { reason }` What a `rejected` entry records. - `reason: string` Why the result was rejected. - `ExtractReviewSignedOffDetail: object { paths }` What a `flags_signed_off` entry records. - `paths: array of string` JSON Pointers of the flagged fields confirmed as part of approving. - `path: optional string` JSON Pointer of the field it changed. - `value_after: optional unknown` The field's value after the change. - `value_before: optional unknown` The field's value before the change.