Define the Schema, Provenance, Versions, and Unknowns

What you will complete

Complete a field dictionary and normalized record with identity, source, extraction, validation, review, destination, and audit fields.

Design one internal record

List only the fields required to make the next decision or create the controlled handoff. Divide them into identity, extracted values, derived values, provenance, validation, review evidence, destination receipt, and audit state. An extractor payload can map into this record. It should not define the record.

For an invoice, identity may include vendor, invoice number, document date, and source hash. Extracted values include currency, line items, subtotal, tax, and printed total. Derived values include calculated line sums or normalized dates. Workflow fields include exception codes, duplicate result, review status, actor, decision time, destination attempts, and destination ID.

Required does not mean “invent when absent.” It means the workflow cannot complete the relevant stage without a value or documented override. Store a missing invoice number as null and route it under the exception policy.

Keep source and normalized values together

A reviewer must be able to trace a value to the original file, page, region, line, or source string. Preserve the supplier’s printed total even when arithmetic conflicts. A normalized date or vendor identifier belongs beside the source value, not over it. Record what changed, why, who approved the correction, and when.

Useful provenance fields include source channel, message or path reference, filename, MIME type, received time, source hash, page or span, extraction adapter, model or API version, and processing run. These fields support investigation and reruns. They do not certify authenticity.

Bind decisions to a document version. A replacement file, changed attachment, or edited record can invalidate an earlier decision. Store the reviewed version or content hash with the approval evidence. If the current version differs, mark the decision stale and begin another cycle.

Worked example: normalize INV-TP-2401

Fictional example: Cedar & Quay records vendor “Ridgemont Industrial Supply Co,” invoice number “INV-TP-2401,” date “2026-09-08,” currency “CAD,” subtotal 400.00, tax 52.00, total 452.00, purchase order “PO-CQ-188,” and three line items. The original values stay linked to their source text.

The normalized record also keeps a source SHA-256, intake ID, received time, filename, adapter name, validation results, duplicate status, review decision, destination attempts, write count, draft ID, and audit-event IDs. The tax label is recorded as printed fixture vocabulary. The workflow does not decide whether the tax treatment is correct.

If the currency symbol appears without an explicit code, the record stores the source symbol and leaves normalized currency unknown. It does not infer CAD from the fictional buyer’s location.

Complete Workbook Section 2

1. List required and optional fields for the next decision.
2. Mark each field extracted, derived, looked up, or entered by a reviewer.
3. Name its type, source pointer, original-value field, normalized-value field, and unknown representation.
4. Add document, file, and business identity keys.
5. Add version, extraction, review, destination, and audit evidence.

Use Section 2 of the workbook.

Decision rules

• Do not force an unknown document type into a known schema.
• Do not overwrite a printed value to make validation pass.
• Do not approve one version and write another.
• Do not make a vendor-specific field name your permanent business contract.
• Do not treat a hash as proof that a supplier or document is genuine.

Module 3 will apply confidence, deterministic validation, and duplicate checks to this record without confusing those signals.

Check your application