Arabic handwriting recognition
Measured across four handwritten corpora: a 14.50% character error rate, against 22.00% for the strongest system compared. The harder the material, the wider the margin.
Handwritten Arabic is where most OCR systems stop working. Not degrade — stop. In the KITAB-Bench measurements, several widely deployed engines exceed a 75% character error rate on handwriting, and one exceeds 600%. If your archive is handwritten, the difference between systems is not a matter of a few percentage points.
Systems tuned for printed Arabic do not transfer
This is the single most useful finding for anyone evaluating OCR for a handwritten archive. Compare each system's performance on the full image-to-text task against its performance on the handwritten subsets:
| System | Full task | Handwriting | Change |
|---|---|---|---|
| YaiGlobal Parser | 12.50% | 14.50% | +2.00 pts |
| Gemini 2.0 Flash | 13.00% | 22.00% | +9.00 pts |
| AIN-7B | 20.00% | 26.00% | +6.00 pts |
| GPT-4o | 31.00% | 40.00% | +9.00 pts |
| Azure Document Intelligence OCR | 52.00% | 76.50% | +24.50 pts |
| Tesseract | 54.00% | 75.75% | +21.75 pts |
| PaddleOCR | 79.00% | 80.50% | +1.50 pts |
| Qwen2.5-VL-7B | 120.00% | 160.75% | +40.75 pts |
| Surya | 495.00% | 610.75% | +115.75 pts |
The parser's position improves relative to the field as the material gets harder. Its margin over the strongest baseline widens from 3.85% on the aggregate task to 34.09% on handwriting — more than eight times. That pattern indicates handwriting capability that is trained for, rather than a by-product of general text recognition.
Error rates above 100% are not typos. Character error rate is a normalized edit distance and is not bounded above — it exceeds 100% when a system emits more spurious content than there was reference text. A system at 610.75% is not reading the page; it is generating text. See the glossary.
What was measured
Four handwritten corpora within KITAB-Bench, covering materially different handwriting conditions:
| Corpus | Material |
|---|---|
| KHATT | Offline handwritten Arabic text, paragraph and line splits |
| ADAB | Isolated handwritten Arabic words |
| Muharaf | Handwritten historical manuscript material |
| OnlineKhatt | Online (stroke-captured) Arabic handwriting |
One anomaly worth flagging, because it works against us. On the ADAB corpus, AIN-7B reports a 0.00% character error rate. That is difficult to explain other than by overlap between its training data and the benchmark. We have no visibility into the training corpora of any baseline. Where such a result inflates a baseline, our reported margin is understated — we flag it rather than rely on it.
What a 14.50% character error rate actually means
It means roughly one character in seven requires review. That is a workable number for human-in-the-loop correction of an archive that was previously not machine-readable at all. It is not a workable number for unattended automation, and we would rather say so here than have you discover it during a pilot.
For comparison, on modern printed Arabic the same system records materially lower error. Handwriting is harder for everyone, including us. The relevant question is not whether a system is perfect on handwriting — none is — but how much of your archive becomes searchable, and how much correction effort that costs.
Who this is for
- National libraries and archives holding handwritten Arabic collections that are catalogued but not searchable.
- Manuscript digitisation programmes where images exist and text does not.
- Academic collections — Islamic studies, Arabic literature, regional history — where the bottleneck is transcription capacity.
- Records offices with handwritten administrative or legal material predating digital systems.
Handwriting is rarely the only problem
Handwritten archives are usually also structural problems: marginalia, tables ruled by hand, annotations across a gutter, footnotes in a different hand. Recognition alone produces a flat transcription that loses all of it.
On the same benchmark the parser records TEDS 97.99 on table structure and an 0.801 layout F1 at 0.916 precision, so structure is recovered alongside text. One caveat worth stating: caption recall is 0.428 — roughly six captions in ten are not detected as captions. For illustrated manuscript material, that specific weakness matters, and it is on our list to fix. The full layout numbers →
Send us your hardest page
Benchmarks are averages over someone else's documents. Send us the worst page in your collection — the water-damaged folio, the crossed-out marginalia, the hand nobody on your team can read — and we will run it and send back the output next to the image, with no obligation and no sales call attached.
If it fails, we will tell you that too. We would rather know now than after a pilot.
Scope of these figures. Vendor-conducted evaluation on the four handwritten corpora within the KITAB-Bench image-to-text track, reported as unweighted macro averages following the published benchmark convention. Baseline figures are quoted from the published KITAB-Bench paper (Heakl et al., Findings of ACL 2025, arXiv:2502.14949). No independent party has replicated this evaluation. Full methodology and limitations.