Donut

by NAVER

Open SourceSelf-HostedMIT

OCR-free document understanding transformer that directly converts document images to structured output without explicit text recognition.

OCRData ExtractionClassification
🤗 Hugging FaceGitHub📄 arXiv Paper

Overview

Donut (Document Understanding Transformer) takes a radically different approach to document processing. Instead of the traditional pipeline of OCR followed by NLP, Donut uses an end-to-end encoder-decoder architecture that directly transforms document images into structured sequences.

The encoder is a Swin Transformer that processes the document image, while the decoder (based on BART) generates structured output in a task-specific format. This eliminates OCR errors from propagating downstream and removes the need for expensive OCR preprocessing.

Donut achieves competitive results on document classification, parsing, and VQA tasks while being simpler to deploy and faster at inference time than OCR-dependent pipelines.

Strengths

  • No OCR dependency - end-to-end processing
  • Simpler deployment pipeline
  • Competitive accuracy on benchmarks
  • Faster inference than OCR + NLP pipelines
  • Handles multiple languages without separate OCR

Limitations

  • Requires task-specific fine-tuning
  • Less interpretable than explicit OCR
  • May hallucinate text in complex documents
  • Lower accuracy than specialized models on some tasks

Best Use Cases

  • Receipt and invoice parsing
  • Document classification
  • Rapid prototyping without OCR setup
  • Multilingual document processing