Invoice Processing Test Pack
Five fictional invoice cases, expected results, schemas, local tests, and an editable scorecard for checking workflow safety before accounting.
Five fictional invoices expose failures that happy-path demonstrations miss: duplicates, missing invoice numbers, conflicting totals, absent reviewers, and retries after a lost response.
Download the free test pack
Use the PDF beside your test environment or edit the Word workbook during a review session. No email or paid account is required.
- Download the PDF workbook
- Download the editable Word workbook
- Open the runnable technical package on GitHub
- Read the Invoice Processing Automation guide
The technical package includes the fictional fixtures, expected JSON, schemas, a standard-library Python runner, and 21 tests.
What this tests
Extraction accuracy is one score. A dependable invoice workflow must also wait for a person, stop duplicates, hold exceptions, and retry a lost response without creating another draft.
| Case | What it tests | Pass condition |
|---|---|---|
| Clean invoice | Complete extraction and approval | Wait for a person, then create one draft |
| Duplicate invoice | Source and invoice-number checks | Stop before review and destination calls |
| Missing invoice number | Required-field exception | Stay in review; do not write without an explicit override |
| Conflicting total | Arithmetic exception | Flag the conflict; do not write without an explicit override |
| Lost destination response | Idempotent retry | Make two attempts with one idempotency key and one write |
What is included
Five invoices with known answers
Run the same cases against an extractor, n8n workflow, Power Automate flow, or custom system. Compare each result with the supplied expected JSON.
An editable scorecard
Record the extractor, workflow version, reviewer, destination, field results, exception decisions, and retest date in the Word workbook.
A local reference runner
Developers can run python3 run_tests.py. It uses the Python standard library and calls no external service.
Clear safety rules
- Every non-duplicate waits for an explicit human
approvebefore any accounting or ERP draft write. - A matching source SHA-256 or vendor plus invoice number stops before review.
- No reviewer means
pending_reviewand zero destination attempts. - The timeout test retries the same idempotency key: two attempts, one write.
- The pack creates drafts only. It never posts, pays, refunds, or changes supplier banking details.
How to use it
- Compare the clean fixture with its expected JSON.
- Map your extractor output into the included invoice schema.
- Run all five cases through the workflow.
- Score extraction, workflow safety, human review, duplicate handling, and destination reliability separately.
- Fix every safety failure before connecting a live destination.
Limits
Version 0.1.0 uses labeled text fixtures. It tests field mapping and workflow controls. It does not measure OCR performance on scans, photos, handwriting, or production invoices.
No Azure, OpenAI, Anthropic, Slack, accounting, ERP, or Supabase service was called to produce the reference results. All buyers, suppliers, addresses, invoice numbers, and amounts are fictional.
When the test exposes a system problem
Flowgrammer builds documented invoice intake-to-accounting systems with human approval, duplicate controls, monitoring, and handover. If the intake channel, reviewer, and accounting draft object are already defined, scope an AI Automation System. If several workflows compete for attention, start with an AI Success Audit.
Frequently asked questions
Does this pack include real invoices?
No. Every supplier, buyer, address, invoice, and amount is fictional.
Does it test OCR accuracy?
No. Version 0.1.0 uses labeled text so teams can isolate field mapping and workflow behaviour.
Can I use it with n8n, Power Automate, or custom code?
Yes. Map your workflow output into the included adapter schema, then compare it with the expected result.
Does a clean invoice write directly to accounting?
No. Every non-duplicate invoice waits for an explicit human approval. The reference destination creates a draft only.
Why test a lost response?
A destination can create a draft and lose the response. A careless retry may create a duplicate. This pack checks that both attempts reuse one idempotency key and produce one write.
Is this a production accounting system?
No. It is an acceptance suite and workbook for evaluating part of an invoice workflow.