Somewhere in every operations business there is a person turning documents into rows. Supplier invoices, packing slips, delivery tickets, timesheets, credit applications, inspection forms photographed on a phone in bad light. It is the most mechanical work in the building and the most reliable place to point AI, because the input is unstructured, the output is a schema, and the value is obvious.
It is also where the naive version does the most damage. A system that extracts confidently and writes silently will corrupt your data faster than any human ever could, and it will do it quietly enough that nobody notices for a quarter. The gap between those two outcomes is almost entirely about what happens after the model reads the page.
Why this got easier, specifically
The old approach was template-based: teach the system where on the page each field sits, per vendor, per form. It worked until a supplier changed their invoice layout, and then it silently produced nonsense until someone caught it. Anyone who ran one of these remembers maintaining hundreds of templates.
Modern models read a document more like a person does — they use the layout as a cue rather than a coordinate map, so "the number next to the word Total" survives the number moving. That is the actual change. It does not mean the output is trustworthy; it means you no longer maintain a template per vendor, which is what made this category impractical before.
The accuracy math nobody does
Vendors quote accuracy per field. Your business cares about documents. Those are very different numbers, and the gap compounds.
| Per-field accuracy | 6 fields | 12 fields | 25 fields |
|---|---|---|---|
| 95% | 74% | 54% | 28% |
| 98% | 89% | 78% | 60% |
| 99.5% | 97% | 94% | 88% |
Read the middle cell again. A model performing at 95% per field on a twelve-field invoice produces a fully correct document about half the time. That is not a bad model — 95% per field is respectable. It is arithmetic, and it is why "our extraction is 95% accurate" tells you almost nothing about how much human review you are about to need.
Not all fields are equally hard
| Field type | Difficulty | Why |
|---|---|---|
| Printed totals and dates | Easy | Distinctive format, labelled, usually machine-set |
| Vendor and document numbers | Easy to moderate | Confusable with other reference numbers on the page |
| Line items in a clean table | Moderate | Row alignment matters; multi-line descriptions break it |
| Line items across a page break | Hard | Continuation rows, repeated headers, running subtotals |
| Handwriting | Hard | Highly variable; numerals are safer than free text |
| Faded thermal receipts and bad photos | Hard | Information is genuinely absent, not misread |
This table is worth building for your own document mix before anything else, because it tells you where the review queue will actually spend its time. In most operations work, the totals and dates are nearly free and the line items are the entire project.
The pipeline
01
Normalize the input
Every source — emailed PDF, scanner output, phone photo, fax — becomes one document object with the file, the source, the sender, and a timestamp. Deskew and correct orientation on images. Photos of paper are a genuinely different input from generated PDFs and it is worth branching on that early.
02
Extract against an explicit schema
Not "read this invoice" but a typed schema with required and optional fields. Every extracted value carries a confidence and, where possible, the location on the page it came from — that provenance is what makes review fast later.
03
Validate deterministically
This is the layer that earns its keep. Line items must sum to the subtotal. Subtotal plus tax must equal the total. Dates must parse and fall in a plausible range. Item codes must exist in your catalog. The vendor must be one you have. None of this involves a model.
04
Route on the result
Everything valid and confident goes through. Anything that failed a check, or has a low-confidence value in a required field, goes to review — flagged with which specific check failed, not just "uncertain".
05
Review with the document beside the fields
The reviewer sees the extracted values next to the page, with the source region highlighted for whichever field they are on. They correct, they confirm, it commits.
06
Commit through the same path as manual entry
Extracted records go through the identical validation and write path a human-entered record would. There is no privileged lane for machine-produced data.
Why the validation layer beats the confidence score
Confidence tells you the model was unsure. It does not tell you the model was wrong, and those are different failures. A model can read a clearly printed 1,240.00 as 1240.00 with total confidence and be right, or read a smudged 7 as a 1 with total confidence and be wrong. High confidence is not a correctness guarantee.
Cross-field arithmetic has no such problem. If the line items sum to $4,180 and the stated total is $4,180, several independently extracted numbers agree with each other — which is far stronger evidence than any of them being individually confident. If they disagree, something is wrong even when every field reports high confidence.
A model can be confidently wrong. Arithmetic cannot. Build the check that does not depend on the thing you are checking.
The general form: for each document type, find the internal consistencies that must hold — sums, date orderings, quantities against a purchase order, codes against a catalog — and encode them as plain deterministic checks. In most workflows these catch more real errors than confidence thresholds do, and they catch the dangerous kind.
The review screen is the ROI
Teams spend months on model selection and an afternoon on the review interface. That is backwards. If straight-through processing is 70%, then 30% of documents are being handled by a person, and whether that takes them fifteen seconds or three minutes is the difference between the project paying for itself and not.
- Document and fields on one screen. Never make someone open the PDF separately. This single thing accounts for most of the time difference.
- Highlight the source region for the focused field, so verifying is a glance instead of a search.
- Land the cursor on the first problem field, not the top of the form. The reviewer should not hunt for what needs attention.
- Keyboard-first. Tab, type, enter. A mouse-driven review screen costs several seconds per field, every time, forever.
- Show what failed and why. "Line items sum to 4,180 but total reads 4,780" is instantly actionable. "Low confidence" is not.
Your review queue is your evaluation set
Every correction a reviewer makes is a labeled example: this document, this field, the model said X, the truth was Y. Store them. Within a few weeks of normal operation you have a regression suite drawn from your own document mix, which is worth considerably more than any public benchmark.
Run it whenever anything changes — the prompt, the model version, the schema. Model updates are not always improvements for your specific documents, and without this you find out from a reviewer three weeks later rather than from a failing test the same day.
What to measure
- Straight-through rate. The share of documents committed with no human touch. The headline number.
- Correction rate per field. The diagnostic one. It names the specific field to fix rather than telling you accuracy is 91%.
- Seconds per exception. Trending up means the review screen or the flagging logic needs work.
- Escaped errors. Wrong data that got committed, found later. Rare, serious, and the only metric worth waking up for.
- All-in cost per document. Inference plus review labor, against what manual entry cost. This is the number the business actually asked about.
Being honest about the limits
Handwriting varies enormously — numerals in boxed fields are usable, free-form cursive on a carbon copy often is not. Tables spanning page breaks remain genuinely hard. A photo taken in a dark truck cab at an angle may simply not contain the information, and no model recovers what was never captured. And any document where a single wrong digit has legal or financial consequence deserves a human confirmation step regardless of how good the numbers look.
The honest framing is that this removes most of the typing and leaves the judgment. That is a large win — it is just not the same as removing the person, and a proposal that promises otherwise is one to be careful with.
Where the extracted data has to land in an older system that has no API, the write path is its own problem — covered in how to connect a legacy system to AI without replacing it.
Frequently asked questions
How accurate is AI document data extraction?
Per-field accuracy is commonly in the mid-to-high nineties, but that is not the number that matters. On a twelve-field document, 95% per-field accuracy yields a fully correct document only about half the time. What determines the outcome is the validation layer that catches the wrong fields automatically.
How is this different from traditional OCR?
Traditional OCR required a template per document layout and broke silently whenever a vendor changed their form. Modern models use layout as a cue rather than a coordinate map, so no per-vendor templates are needed. The extraction still requires validation before anything is committed.
Can it handle handwriting and phone photos?
Partly. Handwritten numerals in boxed fields work reasonably well; free-form cursive is unreliable. Photos are usable when the information is actually legible in the image — a dark, angled shot of a faded thermal receipt may not contain the data at all, and no model can recover what was never captured.
What stops bad data from reaching our system?
Deterministic cross-field checks: line items must sum to the subtotal, dates must parse and be plausible, item codes must exist in your catalog. These catch confidently wrong values that confidence scores miss, and anything failing them is routed to a human with the specific failure shown.