OCR and handwriting accuracy calculator
Measure recognition on a sample of your own pages, then see what the error rate means in mistakes per page and hours of correction.
Calculator
Text recognition output always contains mistakes. The useful questions are how many, on which kinds of page, and what they cost to fix. The calculator above takes the errors you counted on a sample, checked against a careful human transcription, and turns them into a character error rate, a word error rate and an estimate of correction time for the whole batch. It works the same way for OCR of print and typescript and for handwritten text recognition (HTR).
Character error rate and word error rate
- Character error rate (CER) = (substitutions + deletions + insertions) ÷ characters in the reference transcription. A substitution is a wrong character, such as “c” read as “e”. A deletion is a character the output missed. An insertion is a character the output added. Because insertions count, CER can go above 100 percent when the output is mostly noise.
- Character accuracy is 1 minus the CER.
- Word error rate (WER), as this calculator uses it, is the number of reference words with at least one error divided by the number of words in the reference. One wrong letter spoils a whole word, so WER usually runs several times higher than CER. Some evaluation software also counts extra words the engine invents; this hand count leaves those out.
From those counts the calculator works out errors per page (errors ÷ sample pages) and correction time (pages in the batch × errors per page × seconds per fix ÷ 3,600).
How to build a reference sample
- Choose typical pages. Pick pages that look like the batch, not the cleanest ones. If the batch mixes print and handwriting, or several hands, measure each group separately.
- Pick at random within each group, so nobody steers the sample toward easy or hard pages. The QC sample size calculator explains random selection.
- Transcribe exactly. Keep the writer’s spelling, abbreviations and punctuation; do not correct them. Agree rules first for struck-through words, superscripts, the long s and illegible passages. A second person should check the transcription, because it is the yardstick for everything else.
- Decide what counts. Settle whether spaces and punctuation count as characters, and use the same rule for the error counts and the reference count.
- Compare and count. For two or three pages a careful side-by-side reading works. For more, a comparison script is faster.
- Keep training pages out. If a model was trained on some of your pages, measure it on pages it has not seen. Transkribus measures the CER it shows for a model on validation pages not used in training, for the same reason.
Reading the results
There is no official pass mark. Transkribus offers a rule of thumb in its own documentation: a CER of 10 percent or below “can be considered very efficient for automated transcription”, 20 to 30 percent can be enough “to work with powerful searching tools”, and a good CER is about 0.5 to 2 percent for print and 2 to 8 percent for handwriting, depending on whether the model was trained on one hand or many. Treat those as a vendor’s guidance, not a standard.
Errors per page are easier to picture than a percentage. WER tells you more about search than CER does: a single wrong letter in a surname hides that person from an exact search.
Limits
- Correction time is a floor. It covers fixing each error, not reading every line to find it. Full correction means proofreading each line against the image, which usually takes longer than the fixes themselves.
- Small samples are rough. Two pages give a first figure; more pages, drawn at random, narrow it.
- One average hides the worst pages. A single difficult hand can carry most of the errors in a batch.
- Layout is not measured. Merged newspaper columns or ledger entries read in the wrong order do not show up as character errors but can still make text hard to use.
Worked example
Example (fictional). Volunteers at the Aster County Historical Society transcribe two typical pages from Box 3 by hand: one of Ida Okerlund’s letters and one page of the Wren Gulch Mercantile day book. Together the pages hold 2,400 characters and 430 words. Handwriting recognition substitutes 38 characters, misses 12 and adds 9, for 59 errors.
| Measure | Arithmetic | Result |
|---|---|---|
| Character error rate | 59 ÷ 2,400 | 2.46% |
| Character accuracy | 100% − 2.46% | 97.54% |
| Word error rate | 31 ÷ 430 words | 7.21% |
| Errors per page | 59 ÷ 2 pages | 29.5 |
| Correction time, 1,200 pages | 1,200 × 29.5 × 6 s ÷ 3,600 | 59.0 hours |
A CER of 2.46 percent sits inside the range Transkribus calls good for handwriting, yet about one word in fourteen still contains an error. The society chooses targeted correction of names, places and dates for its 1,200 pages of letters and day books in the same hands.
Text recognition describes how Dastok measures error rates per batch and per kind of page, and the correction levels you can choose. Our AI policy explains how uncorrected text stays labeled as machine output, and the fictional example collection shows recognized text feeding collection search. The Reading Room’s text recognition section goes further into OCR and handwriting.
Send one box first.
A pilot box shows you the images, the recognized text and the catalog records before you commit a whole collection. Tell us what is in the box and what you want people to find.
Studio and contacts
24 South Weber Street, Suite 225Colorado Springs, CO 80903(719) 244-8450sales@dastok.com