Skip to content
CanCode DevsCanCode Devs
All work

Case study

End-to-End Document Intelligence Pipeline

In plain English

A factory of specialist workers that turns messy medical scans into structured cards and summaries.

Reached 84% end-to-end accuracy, later 88% with an LLM extraction layer.

Who it is for

  • Operations teams processing high volumes of medical and insurance paperwork.
  • Engineering buyers who need modular CV and OCR services they can retrain and scale one queue at a time.
End-to-End Document Intelligence Pipeline screenshot
End-to-End Document Intelligence Pipeline screenshot

Problem

Medical and insurance packets arrive as poor scans: mixed print and handwriting, ticks, skew, multi-page claims. A single model that “reads the page” cannot be retrained for handwriting without touching billing.

Each failure mode needs a specialist, and work has to move as jobs on queues, not a notebook chained in docker-compose.

Solution

Each page is a JSON job on a FIFO queue. Independent Docker workers read object-storage artifacts, write the next prefix, and record work items. Geometric preprocess deskews and labels noise with a four-class CNN. Known forms align and match fields; other pages go through page OCR (docTR) and crop OCR (trOCR), then spellcheck, clinical cards, and a human-readable summary.

A later LLM extraction layer consumes the same OCR words. Production also runs additional GPU workers; this case study covers the factory pattern and the stages we ship as independent images.

Outcomes

Reached 84% end-to-end accuracy in production (engagement-reported).

The same artifacts let an LLM extraction layer raise accuracy to 88%.

PyTorchOpenCVTransformersdocTRtrOCRSQSS3DynamoDBKubernetesDocker

How it works

A page job on FIFO queues, on-disk stages from geometric preprocess through OCR and spellcheck, then medical cards and a summary.
A page job on FIFO queues, on-disk stages from geometric preprocess through OCR and spellcheck, then medical cards and a summary.

Template path vs unstructured

After geometric preprocess, known forms align and match fields; other pages go OCR, then both paths spellcheck into cards.
After geometric preprocess, known forms align and match fields; other pages go OCR, then both paths spellcheck into cards.

What we built

Document factory

  • FIFO jobs

    Orchestrator enqueues a page; a worker long-polls, writes the next artifact, records the work item.

  • Geometric preprocess

    Deskew, crop, and a four-class noise label.

  • Template vs unstructured

    Known forms warp onto a template and match named fields. Other pages skip the template and go OCR.

  • OCR, repair, cards

    docTR and trOCR, then field repair, clinical cards, and an Excel-plus-PDF summary.

How it works

Callers are queue workers, not a staff web app. Each stage is its own image. Helm replica counts are often zero; an autoscaler sets replicas from queue depth because cloud queue metrics lag.

Retrain OCR without touching cards. Failures stay on their queue. The LLM extraction layer sits downstream on the same words.

Integrations

  • SQS, S3, DynamoDB

    FIFO jobs, page artifacts, and work-item metadata.

  • Kubernetes

    One image per stage; workers scale on queue depth.

  • LLM extraction layer

    Consumes the same OCR words and boxes; see Medical Document Entity Extraction.

Have a similar challenge?

Start a project