Document Automation Testing Guide
Seven control types, expected outputs, and one destination write. OCR is not measured.
— Craig Major
Direct answer
A document-automation test pack names expected outputs, exception cases, and one destination write before a workflow goes live. Every document family needs clean, duplicate, missing-field, conflict, human-review, retry, and idempotency cases. Labelled-text fixtures do not measure OCR. The live Invoice Processing Test Pack stays the invoice example. This page is the reusable method.
Use this when you must prove a document workflow before it writes a draft. Start with Document Processing Automation if you are still choosing what to automate, or Intelligent Document Processing for extraction vocabulary.
Who this is for
Operations or QA leads who own a document family and a draft destination. Prerequisites: one named reviewer, one expected-output schema, and fictional or approved files. You do not need a ranked platform shortlist to fill the template.
Seven control types
Every family, not only invoices, should record:
- Clean: required fields present, one later draft after review
- Duplicate: same source bytes or same business key
- Missing fields: required field blank
- Conflict: two values that cannot both be true
- Human review: a person must decide
- Destination retry: first response lost
- Idempotency: second write blocked
OCR accuracy is not a control type on labelled-text fixtures. If you later add a scan corpus, keep OCR scores on a separate sheet. Do not mix them into this pack.
Worked example
Cedar and Quay Fabrication Ltd already has five published invoice cases in the Invoice Processing Test Pack: INV-TP-2401 clean, a byte-identical duplicate, a missing invoice number, a conflicting total, and INV-TP-2405 destination timeout.
This template maps those five cases as examples only. It does not replace that live pack. Then it adds a blank packing-slip family with the same seven control types and empty expected fields. Avery Chen still owns the human-review row. Payment destinations stay off the sheet.
Workflow
- Name the document family and the draft destination.
- Copy the expected-output schema. List required fields.
- Add one row for each of the seven control types.
- Attach or describe a fictional fixture for each row.
- Record expected status, reasons, review invocations, destination attempts, and
write_count. - Run the workflow or the offline harness.
- Mark pass, fail, or blocked. A person judges exceptions.
- Do not publish a family that skipped duplicate, retry, or idempotency.
Decision table
| Signal | Automatic | Review | Human-only |
|---|---|---|---|
| Expected schema missing | Fail the pack | None | Author the schema |
| Clean case | Record fields | Approve before draft | Confirm destination |
| Duplicate | Stop | None | Override policy |
| Missing or conflict | Flag | Person decides | Do not invent values |
| Retry after timeout | Same key | Confirm one write | Do not force a second post |
| OCR score requested on labelled text | Reject the claim | None | Build a scan corpus later |
| Payment destination added | Block the row | None | Keep payment outside this pack |
Human gates
A person sets expected fields. A person judges exceptions. A person confirms that a retry did not create a second write. Avery Chen is the fictional named reviewer in the sample rows.
Failure paths
- Missing expected-output schema: the pack is not ready
- Skipped duplicate case: do not treat the family as tested
- OCR treated as measured on labelled text: honesty fail
- Payment destination added: remove it
- Replacing the live invoice pack route with this template: do not
- Idempotency row missing
write_count=1: fail
Test cases
| Case | Input | Expected | Acceptance |
|---|---|---|---|
| Clean | Complete fictional document | Pending review, then one draft | Schema fields filled |
| Duplicate | Same bytes or business key | Stop, zero writes | Duplicate reason recorded |
| Missing fields | Required field blank | Exception, no draft | Field named |
| Conflict | Two incompatible values | Exception, no draft | Conflict named |
| Human review | Non-duplicate exception or clean | Reviewer decide | Named person |
| Retry | Destination timeout | Attempts greater than writes | Same key |
| Idempotency | Second post after success | write_count=1 |
No second draft |
The packing-slip blank rows use the same acceptance rules with empty fixtures until a buyer fills them.
Cost and measurement
This template does not price seats or OCR. If you later measure scan quality, record model, date, and corpus separately. Do not invent accuracy percentages here.
Asset instructions
Download Document Automation Test Pack. Open the XLSX Cover sheet. Read the OCR-not-measured and no-payment banners. Fill ExpectedOutput and CaseRegister. CSV and Markdown copies ship in the same ZIP. Formulas count cases. They do not score OCR.
Cross-check invoice examples against the live Invoice Processing Test Pack. Use the IDP Requirements and Evaluation Worksheet when you are choosing an extractor, not when you are proving workflow gates.
Next step
If the test pack shows missing owners or destinations, start with a Document Workflow Opportunity Audit. If cases already pass and you want a build, book a fit call for AI Automation Systems. Use Book a fit call when you are ready.