Online Urdu Text-Line Recognition

Turning pen dynamics into image channels so pretrained offline recognizers can use them. First-author paper at ICDAR 2026.

Ali Hussain, Rafay Ahmad, Momina Moetesum, Adnan Ul-Hasan, Faisal Shafait. ICDAR 2026. Paper (PDF) · Code · OUHD-L Dataset

The problem

Urdu is written in Nastaliq, one of the hardest scripts in active use. It runs right to left, its baseline descends diagonally, its glyphs overlap heavily, and it has more than 24,000 context-dependent ligature forms. Digital pens record rich online signals (pressure, tilt, timing), but there was no public line-level online Urdu dataset. There was also far too little labeled data to train online sequence models or fine-tune large VLMs without overfitting.

Pen pressure separates ligatures that look almost identical as static ink.

What I did

  1. Built OUHD-L, the first public online Urdu text-line dataset: 2,403 lines from 311 writers, captured on a Wacom IoT Paper device at 500 Hz. It records x, y, pressure, tilt and timestamps, uses writer-disjoint splits, and contains 5,829 unique ligatures.
  2. Encoded online dynamics as images. I rendered 11 auxiliary channels (Δx/Δy, velocity, acceleration, curvature, direction sin/cos, pressure, tilt, global stroke order), each pixel-aligned with the ink.
  3. Fused them into a pretrained offline recognizer. Channel-adaptive and dynamic gated fusion inject these channels into a pretrained Urdu text-line model, so the sequence model is reused rather than retrained from scratch.
The raw trajectory (x, y, p, α, β, t) becomes a 12-channel image tensor. Per-channel CNNs are fused by learned gates before a Transformer decoder.

Results

Model CER (%)
Pretrained offline model, zero-shot on rendered ink 10.65
Ink-only fine-tuning 4.21 ± 0.09
Full online-feature fusion (ours) 3.66 ± 0.04
Qwen3-VL-4B + QLoRA 7.28
DeepSeek-OCR 38.04
  • Fusion gives a 13% relative CER reduction over five seeds (paired t-test, t(4) = 14.45, p < 0.001).
  • Every channel helps on its own. Direction cosine helps most, because heading angle at ligature junctions is the key cue in Nastaliq.
  • General-purpose VLMs fall well short, even after fine-tuning. For low-resource cursive scripts, a better representation beats a bigger model.

Why it matters

The recipe does not depend on the script. It lets any language with mature offline recognizers take advantage of pen data without a large online corpus. This motivates my ongoing work on recovering online trajectories from offline images.