# Review ## Create Extract Review `extract.review.create(strjob_id, ReviewCreateParams**kwargs) -> ExtractReview` **post** `/api/v2/extract/{job_id}/review` Beta: answers 403 until extract review is enabled for the caller and their organization. Turn review on for an extract job. Fields read below `confidence_threshold`, and required fields left empty, are flagged for a person to check. Every setting has a default, so the body can be left out. A job still running can be named: it waits for review once it completes. A job that failed or was cancelled without a review, or that cannot be reviewed, answers 409 with `code: job_not_reviewable`; a field in `fields_in_scope` the job's schema does not have answers 400. Answers 201 with the new review, or 200 with the job's existing review, whose settings stay as they are. ### Parameters - `job_id: str` - `organization_id: Optional[str]` - `project_id: Optional[str]` - `confidence_threshold: Optional[float]` Fields read below this confidence are flagged. - `fields_in_scope: Optional[Sequence[str]]` JSON Pointers of the fields the threshold applies to; empty means all. - `required_fields: Optional[Sequence[str]]` JSON Pointers of fields flagged when empty; defaults to the schema's required. ### Returns - `class ExtractReview: …` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: str` Unique identifier of the review. - `confidence_threshold: float` Fields read below this confidence are flagged. - `created_at: datetime` When review was turned on. - `decided_at: Optional[datetime]` When it was approved or rejected. - `decided_by: Optional[str]` ID of the user who approved or rejected it. - `decided_reason: Optional[str]` Why it was rejected. - `field_actions: Dict[str, FieldActions]` The latest action per field, keyed by JSON Pointer. - `action: Literal["clear", "confirm", "correct"]` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: str` ID of the user who acted. - `at: datetime` When they acted. - `value: Optional[object]` The corrected value, for `correct`. - `fields_in_scope: List[str]` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: List[Flag]` Fields the review asks a person to check. - `handled: bool` Whether someone confirmed, corrected or cleared it. - `path: str` JSON Pointer of the field. - `reason: Literal["below_threshold", "required_empty"]` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: Optional[FlagCell]` Where in a list the least confident value is. - `row: int` Index of the row, from 0. - `column: Optional[str]` Property of the row, when rows are objects. - `confidence: Optional[float]` The field's confidence; for a list, its least confident value. - `job_id: str` The extract job reviewed. - `open_flags: List[str]` JSON Pointers of flagged fields nobody checked. - `original_result: Optional[Dict[str, object]]` The result as the model read it. - `required_fields: List[str]` JSON Pointers of fields flagged when empty. - `retain_until: datetime` When the job and its document may be deleted. - `reviewed_result: Dict[str, object]` The result with the field actions. - `revision: int` Bumped by every change; actions name the one they saw. - `source_file_id: Optional[str]` The file the job read, or the file behind the parse job it read; null for a URL. - `status: Literal["approved", "pending", "rejected"]` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: datetime` When the review last changed. ### Example ```python import os from llama_cloud import LlamaCloud client = LlamaCloud( api_key=os.environ.get("LLAMA_CLOUD_API_KEY"), # This is the default and can be omitted ) extract_review = client.extract.review.create( job_id="job_id", ) print(extract_review.id) ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## Get Extract Review `extract.review.get(strjob_id, ReviewGetParams**kwargs) -> ExtractReview` **get** `/api/v2/extract/{job_id}/review` Beta: answers 403 until extract review is enabled for the caller and their organization. Get an extract job's review: its flags, field actions and decision, with the result as the model read it and as reviewed. A job without a review answers 404. ### Parameters - `job_id: str` - `organization_id: Optional[str]` - `project_id: Optional[str]` ### Returns - `class ExtractReview: …` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: str` Unique identifier of the review. - `confidence_threshold: float` Fields read below this confidence are flagged. - `created_at: datetime` When review was turned on. - `decided_at: Optional[datetime]` When it was approved or rejected. - `decided_by: Optional[str]` ID of the user who approved or rejected it. - `decided_reason: Optional[str]` Why it was rejected. - `field_actions: Dict[str, FieldActions]` The latest action per field, keyed by JSON Pointer. - `action: Literal["clear", "confirm", "correct"]` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: str` ID of the user who acted. - `at: datetime` When they acted. - `value: Optional[object]` The corrected value, for `correct`. - `fields_in_scope: List[str]` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: List[Flag]` Fields the review asks a person to check. - `handled: bool` Whether someone confirmed, corrected or cleared it. - `path: str` JSON Pointer of the field. - `reason: Literal["below_threshold", "required_empty"]` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: Optional[FlagCell]` Where in a list the least confident value is. - `row: int` Index of the row, from 0. - `column: Optional[str]` Property of the row, when rows are objects. - `confidence: Optional[float]` The field's confidence; for a list, its least confident value. - `job_id: str` The extract job reviewed. - `open_flags: List[str]` JSON Pointers of flagged fields nobody checked. - `original_result: Optional[Dict[str, object]]` The result as the model read it. - `required_fields: List[str]` JSON Pointers of fields flagged when empty. - `retain_until: datetime` When the job and its document may be deleted. - `reviewed_result: Dict[str, object]` The result with the field actions. - `revision: int` Bumped by every change; actions name the one they saw. - `source_file_id: Optional[str]` The file the job read, or the file behind the parse job it read; null for a URL. - `status: Literal["approved", "pending", "rejected"]` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: datetime` When the review last changed. ### Example ```python import os from llama_cloud import LlamaCloud client = LlamaCloud( api_key=os.environ.get("LLAMA_CLOUD_API_KEY"), # This is the default and can be omitted ) extract_review = client.extract.review.get( job_id="job_id", ) print(extract_review.id) ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## Change Extract Review Fields `extract.review.change_fields(strjob_id, ReviewChangeFieldsParams**kwargs) -> ExtractReview` **post** `/api/v2/extract/{job_id}/review/fields` Beta: answers 403 until extract review is enabled for the caller and their organization. Confirm, correct, clear, restore or undo fields of a completed job. The changes apply in order, in one write. A job without a review gets one with the default settings, at revision 0. A path that is not a field of the job, or a value the field's schema refuses, answers 400 and changes nothing. A job that has not completed answers 409 with `code: job_not_reviewable`. A `revision` other than the review's latest answers 409 with `code: stale_revision` and the latest `revision`. ### Parameters - `job_id: str` - `changes: Iterable[Change]` Changes to apply, in order. - `change: Literal["clear", "confirm", "correct", 2 more]` confirm, correct to `value`, clear, restore the model's value, or undo. - `"clear"` - `"confirm"` - `"correct"` - `"restore"` - `"undo"` - `path: str` JSON Pointer of the field. - `value: Optional[object]` The corrected value, for `correct`. - `revision: int` The review's revision as the caller last read it. - `organization_id: Optional[str]` - `project_id: Optional[str]` ### Returns - `class ExtractReview: …` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: str` Unique identifier of the review. - `confidence_threshold: float` Fields read below this confidence are flagged. - `created_at: datetime` When review was turned on. - `decided_at: Optional[datetime]` When it was approved or rejected. - `decided_by: Optional[str]` ID of the user who approved or rejected it. - `decided_reason: Optional[str]` Why it was rejected. - `field_actions: Dict[str, FieldActions]` The latest action per field, keyed by JSON Pointer. - `action: Literal["clear", "confirm", "correct"]` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: str` ID of the user who acted. - `at: datetime` When they acted. - `value: Optional[object]` The corrected value, for `correct`. - `fields_in_scope: List[str]` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: List[Flag]` Fields the review asks a person to check. - `handled: bool` Whether someone confirmed, corrected or cleared it. - `path: str` JSON Pointer of the field. - `reason: Literal["below_threshold", "required_empty"]` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: Optional[FlagCell]` Where in a list the least confident value is. - `row: int` Index of the row, from 0. - `column: Optional[str]` Property of the row, when rows are objects. - `confidence: Optional[float]` The field's confidence; for a list, its least confident value. - `job_id: str` The extract job reviewed. - `open_flags: List[str]` JSON Pointers of flagged fields nobody checked. - `original_result: Optional[Dict[str, object]]` The result as the model read it. - `required_fields: List[str]` JSON Pointers of fields flagged when empty. - `retain_until: datetime` When the job and its document may be deleted. - `reviewed_result: Dict[str, object]` The result with the field actions. - `revision: int` Bumped by every change; actions name the one they saw. - `source_file_id: Optional[str]` The file the job read, or the file behind the parse job it read; null for a URL. - `status: Literal["approved", "pending", "rejected"]` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: datetime` When the review last changed. ### Example ```python import os from llama_cloud import LlamaCloud client = LlamaCloud( api_key=os.environ.get("LLAMA_CLOUD_API_KEY"), # This is the default and can be omitted ) extract_review = client.extract.review.change_fields( job_id="job_id", changes=[{ "change": "correct", "path": "/total", }], revision=0, ) print(extract_review.id) ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## Approve Extract Review `extract.review.approve(strjob_id, ReviewApproveParams**kwargs) -> ExtractReview` **post** `/api/v2/extract/{job_id}/review/approve` Beta: answers 403 until extract review is enabled for the caller and their organization. Approve the reviewed result. While flagged fields nobody checked remain, answers 409 with `code: open_flags` and their `paths`, unless `sign_off` confirms them as part of approving. A job that has not completed answers 409 with `code: job_not_reviewable`. A `revision` other than the review's latest answers 409 with `code: stale_revision` and the latest `revision`. ### Parameters - `job_id: str` - `revision: int` The review's revision as the caller last read it. - `organization_id: Optional[str]` - `project_id: Optional[str]` - `sign_off: Optional[bool]` Confirm the flagged fields nobody checked as part of approving. ### Returns - `class ExtractReview: …` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: str` Unique identifier of the review. - `confidence_threshold: float` Fields read below this confidence are flagged. - `created_at: datetime` When review was turned on. - `decided_at: Optional[datetime]` When it was approved or rejected. - `decided_by: Optional[str]` ID of the user who approved or rejected it. - `decided_reason: Optional[str]` Why it was rejected. - `field_actions: Dict[str, FieldActions]` The latest action per field, keyed by JSON Pointer. - `action: Literal["clear", "confirm", "correct"]` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: str` ID of the user who acted. - `at: datetime` When they acted. - `value: Optional[object]` The corrected value, for `correct`. - `fields_in_scope: List[str]` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: List[Flag]` Fields the review asks a person to check. - `handled: bool` Whether someone confirmed, corrected or cleared it. - `path: str` JSON Pointer of the field. - `reason: Literal["below_threshold", "required_empty"]` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: Optional[FlagCell]` Where in a list the least confident value is. - `row: int` Index of the row, from 0. - `column: Optional[str]` Property of the row, when rows are objects. - `confidence: Optional[float]` The field's confidence; for a list, its least confident value. - `job_id: str` The extract job reviewed. - `open_flags: List[str]` JSON Pointers of flagged fields nobody checked. - `original_result: Optional[Dict[str, object]]` The result as the model read it. - `required_fields: List[str]` JSON Pointers of fields flagged when empty. - `retain_until: datetime` When the job and its document may be deleted. - `reviewed_result: Dict[str, object]` The result with the field actions. - `revision: int` Bumped by every change; actions name the one they saw. - `source_file_id: Optional[str]` The file the job read, or the file behind the parse job it read; null for a URL. - `status: Literal["approved", "pending", "rejected"]` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: datetime` When the review last changed. ### Example ```python import os from llama_cloud import LlamaCloud client = LlamaCloud( api_key=os.environ.get("LLAMA_CLOUD_API_KEY"), # This is the default and can be omitted ) extract_review = client.extract.review.approve( job_id="job_id", revision=0, ) print(extract_review.id) ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## Reject Extract Review `extract.review.reject(strjob_id, ReviewRejectParams**kwargs) -> ExtractReview` **post** `/api/v2/extract/{job_id}/review/reject` Beta: answers 403 until extract review is enabled for the caller and their organization. Reject the result, with a reason. A job that failed or was cancelled can be rejected, which closes a review turned on while it ran. A job still queued or running answers 409 with `code: job_not_reviewable`. A `revision` other than the review's latest answers 409 with `code: stale_revision` and the latest `revision`. ### Parameters - `job_id: str` - `reason: str` Why the result was rejected. - `revision: int` The review's revision as the caller last read it. - `organization_id: Optional[str]` - `project_id: Optional[str]` ### Returns - `class ExtractReview: …` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: str` Unique identifier of the review. - `confidence_threshold: float` Fields read below this confidence are flagged. - `created_at: datetime` When review was turned on. - `decided_at: Optional[datetime]` When it was approved or rejected. - `decided_by: Optional[str]` ID of the user who approved or rejected it. - `decided_reason: Optional[str]` Why it was rejected. - `field_actions: Dict[str, FieldActions]` The latest action per field, keyed by JSON Pointer. - `action: Literal["clear", "confirm", "correct"]` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: str` ID of the user who acted. - `at: datetime` When they acted. - `value: Optional[object]` The corrected value, for `correct`. - `fields_in_scope: List[str]` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: List[Flag]` Fields the review asks a person to check. - `handled: bool` Whether someone confirmed, corrected or cleared it. - `path: str` JSON Pointer of the field. - `reason: Literal["below_threshold", "required_empty"]` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: Optional[FlagCell]` Where in a list the least confident value is. - `row: int` Index of the row, from 0. - `column: Optional[str]` Property of the row, when rows are objects. - `confidence: Optional[float]` The field's confidence; for a list, its least confident value. - `job_id: str` The extract job reviewed. - `open_flags: List[str]` JSON Pointers of flagged fields nobody checked. - `original_result: Optional[Dict[str, object]]` The result as the model read it. - `required_fields: List[str]` JSON Pointers of fields flagged when empty. - `retain_until: datetime` When the job and its document may be deleted. - `reviewed_result: Dict[str, object]` The result with the field actions. - `revision: int` Bumped by every change; actions name the one they saw. - `source_file_id: Optional[str]` The file the job read, or the file behind the parse job it read; null for a URL. - `status: Literal["approved", "pending", "rejected"]` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: datetime` When the review last changed. ### Example ```python import os from llama_cloud import LlamaCloud client = LlamaCloud( api_key=os.environ.get("LLAMA_CLOUD_API_KEY"), # This is the default and can be omitted ) extract_review = client.extract.review.reject( job_id="job_id", reason="The invoice total does not match the line items.", revision=0, ) print(extract_review.id) ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## Reopen Extract Review `extract.review.reopen(strjob_id, ReviewReopenParams**kwargs) -> ExtractReview` **post** `/api/v2/extract/{job_id}/review/reopen` Beta: answers 403 until extract review is enabled for the caller and their organization. Put an approved or rejected review back to pending. Its field actions stay. A `revision` other than the review's latest answers 409 with `code: stale_revision` and the latest `revision`. ### Parameters - `job_id: str` - `revision: int` The review's revision as the caller last read it. - `organization_id: Optional[str]` - `project_id: Optional[str]` ### Returns - `class ExtractReview: …` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: str` Unique identifier of the review. - `confidence_threshold: float` Fields read below this confidence are flagged. - `created_at: datetime` When review was turned on. - `decided_at: Optional[datetime]` When it was approved or rejected. - `decided_by: Optional[str]` ID of the user who approved or rejected it. - `decided_reason: Optional[str]` Why it was rejected. - `field_actions: Dict[str, FieldActions]` The latest action per field, keyed by JSON Pointer. - `action: Literal["clear", "confirm", "correct"]` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: str` ID of the user who acted. - `at: datetime` When they acted. - `value: Optional[object]` The corrected value, for `correct`. - `fields_in_scope: List[str]` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: List[Flag]` Fields the review asks a person to check. - `handled: bool` Whether someone confirmed, corrected or cleared it. - `path: str` JSON Pointer of the field. - `reason: Literal["below_threshold", "required_empty"]` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: Optional[FlagCell]` Where in a list the least confident value is. - `row: int` Index of the row, from 0. - `column: Optional[str]` Property of the row, when rows are objects. - `confidence: Optional[float]` The field's confidence; for a list, its least confident value. - `job_id: str` The extract job reviewed. - `open_flags: List[str]` JSON Pointers of flagged fields nobody checked. - `original_result: Optional[Dict[str, object]]` The result as the model read it. - `required_fields: List[str]` JSON Pointers of fields flagged when empty. - `retain_until: datetime` When the job and its document may be deleted. - `reviewed_result: Dict[str, object]` The result with the field actions. - `revision: int` Bumped by every change; actions name the one they saw. - `source_file_id: Optional[str]` The file the job read, or the file behind the parse job it read; null for a URL. - `status: Literal["approved", "pending", "rejected"]` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: datetime` When the review last changed. ### Example ```python import os from llama_cloud import LlamaCloud client = LlamaCloud( api_key=os.environ.get("LLAMA_CLOUD_API_KEY"), # This is the default and can be omitted ) extract_review = client.extract.review.reopen( job_id="job_id", revision=0, ) print(extract_review.id) ``` #### Response ```json { "id": "id", "confidence_threshold": 0, "created_at": "2019-12-27T18:11:19.117Z", "decided_at": "2019-12-27T18:11:19.117Z", "decided_by": "decided_by", "decided_reason": "decided_reason", "field_actions": { "foo": { "action": "clear", "actor_id": "actor_id", "at": "2019-12-27T18:11:19.117Z", "value": {} } }, "fields_in_scope": [ "string" ], "flags": [ { "handled": true, "path": "path", "reason": "below_threshold", "cell": { "row": 0, "column": "column" }, "confidence": 0 } ], "job_id": "job_id", "open_flags": [ "string" ], "original_result": { "foo": "bar" }, "required_fields": [ "string" ], "retain_until": "2019-12-27T18:11:19.117Z", "reviewed_result": { "foo": "bar" }, "revision": 0, "source_file_id": "source_file_id", "status": "approved", "updated_at": "2019-12-27T18:11:19.117Z" } ``` ## List Extract Review Events `extract.review.list_events(strjob_id, ReviewListEventsParams**kwargs) -> SyncPaginatedCursor[ExtractReviewEvent]` **get** `/api/v2/extract/{job_id}/review/events` Beta: answers 403 until extract review is enabled for the caller and their organization. List a review's history, newest first. ### Parameters - `job_id: str` - `organization_id: Optional[str]` - `page_size: Optional[int]` Events per page. - `page_token: Optional[str]` Token from the previous page. - `project_id: Optional[str]` ### Returns - `class ExtractReviewEvent: …` One entry in a review's history. - `id: str` Unique identifier of the entry. - `actor_id: str` ID of the user who acted. - `created_at: datetime` When it happened. - `kind: Literal["approved", "created", "field_cleared", 7 more]` What happened. - `"approved"` - `"created"` - `"field_cleared"` - `"field_confirmed"` - `"field_corrected"` - `"field_restored"` - `"field_undone"` - `"flags_signed_off"` - `"rejected"` - `"reopened"` - `detail: Optional[Detail]` The settings for `created`, the reason for `rejected`, and the fields for `flags_signed_off`; null for the rest. - `class DetailExtractReviewCreatedDetail: …` What a `created` entry records: where review was turned on, and its settings. - `confidence_threshold: float` Fields read below this confidence are flagged. - `fields_in_scope: List[str]` JSON Pointers of the fields the threshold applies to; empty means all. - `origin: Literal["api", "edit", "history", "run"]` Turned on as the job ran, from History, by a first edit, or by the API. - `"api"` - `"edit"` - `"history"` - `"run"` - `required_fields: List[str]` JSON Pointers of fields flagged when empty. - `class DetailExtractReviewRejectedDetail: …` What a `rejected` entry records. - `reason: str` Why the result was rejected. - `class DetailExtractReviewSignedOffDetail: …` What a `flags_signed_off` entry records. - `paths: List[str]` JSON Pointers of the flagged fields confirmed as part of approving. - `path: Optional[str]` JSON Pointer of the field it changed. - `value_after: Optional[object]` The field's value after the change. - `value_before: Optional[object]` The field's value before the change. ### Example ```python import os from llama_cloud import LlamaCloud client = LlamaCloud( api_key=os.environ.get("LLAMA_CLOUD_API_KEY"), # This is the default and can be omitted ) page = client.extract.review.list_events( job_id="job_id", ) page = page.items[0] print(page.id) ``` #### Response ```json { "items": [ { "id": "id", "actor_id": "actor_id", "created_at": "2019-12-27T18:11:19.117Z", "kind": "approved", "detail": { "confidence_threshold": 0, "fields_in_scope": [ "string" ], "origin": "api", "required_fields": [ "string" ] }, "path": "path", "value_after": {}, "value_before": {} } ], "next_page_token": "next_page_token", "total_size": 0 } ``` ## Domain Types ### Extract Review - `class ExtractReview: …` An extract job's review: its settings, flags, field actions and decision, with the result as read and as reviewed. - `id: str` Unique identifier of the review. - `confidence_threshold: float` Fields read below this confidence are flagged. - `created_at: datetime` When review was turned on. - `decided_at: Optional[datetime]` When it was approved or rejected. - `decided_by: Optional[str]` ID of the user who approved or rejected it. - `decided_reason: Optional[str]` Why it was rejected. - `field_actions: Dict[str, FieldActions]` The latest action per field, keyed by JSON Pointer. - `action: Literal["clear", "confirm", "correct"]` confirm, correct or clear. - `"clear"` - `"confirm"` - `"correct"` - `actor_id: str` ID of the user who acted. - `at: datetime` When they acted. - `value: Optional[object]` The corrected value, for `correct`. - `fields_in_scope: List[str]` JSON Pointers of the fields the threshold applies to; empty means all. - `flags: List[Flag]` Fields the review asks a person to check. - `handled: bool` Whether someone confirmed, corrected or cleared it. - `path: str` JSON Pointer of the field. - `reason: Literal["below_threshold", "required_empty"]` below_threshold, or required_empty for a required field left empty. - `"below_threshold"` - `"required_empty"` - `cell: Optional[FlagCell]` Where in a list the least confident value is. - `row: int` Index of the row, from 0. - `column: Optional[str]` Property of the row, when rows are objects. - `confidence: Optional[float]` The field's confidence; for a list, its least confident value. - `job_id: str` The extract job reviewed. - `open_flags: List[str]` JSON Pointers of flagged fields nobody checked. - `original_result: Optional[Dict[str, object]]` The result as the model read it. - `required_fields: List[str]` JSON Pointers of fields flagged when empty. - `retain_until: datetime` When the job and its document may be deleted. - `reviewed_result: Dict[str, object]` The result with the field actions. - `revision: int` Bumped by every change; actions name the one they saw. - `source_file_id: Optional[str]` The file the job read, or the file behind the parse job it read; null for a URL. - `status: Literal["approved", "pending", "rejected"]` pending until approved or rejected. - `"approved"` - `"pending"` - `"rejected"` - `updated_at: datetime` When the review last changed. ### Extract Review Event - `class ExtractReviewEvent: …` One entry in a review's history. - `id: str` Unique identifier of the entry. - `actor_id: str` ID of the user who acted. - `created_at: datetime` When it happened. - `kind: Literal["approved", "created", "field_cleared", 7 more]` What happened. - `"approved"` - `"created"` - `"field_cleared"` - `"field_confirmed"` - `"field_corrected"` - `"field_restored"` - `"field_undone"` - `"flags_signed_off"` - `"rejected"` - `"reopened"` - `detail: Optional[Detail]` The settings for `created`, the reason for `rejected`, and the fields for `flags_signed_off`; null for the rest. - `class DetailExtractReviewCreatedDetail: …` What a `created` entry records: where review was turned on, and its settings. - `confidence_threshold: float` Fields read below this confidence are flagged. - `fields_in_scope: List[str]` JSON Pointers of the fields the threshold applies to; empty means all. - `origin: Literal["api", "edit", "history", "run"]` Turned on as the job ran, from History, by a first edit, or by the API. - `"api"` - `"edit"` - `"history"` - `"run"` - `required_fields: List[str]` JSON Pointers of fields flagged when empty. - `class DetailExtractReviewRejectedDetail: …` What a `rejected` entry records. - `reason: str` Why the result was rejected. - `class DetailExtractReviewSignedOffDetail: …` What a `flags_signed_off` entry records. - `paths: List[str]` JSON Pointers of the flagged fields confirmed as part of approving. - `path: Optional[str]` JSON Pointer of the field it changed. - `value_after: Optional[object]` The field's value after the change. - `value_before: Optional[object]` The field's value before the change.