[Feature]: use local (small) vision llm for higer OCR accuracy

[mgcsysinfcat]

Describe the proposed feature

like Qwen2.5-VL 3B or Gamma3 4b eg.

[drindt]

That would be very interesting, but the question is, how do these models deliver their OCR results?
And how do they use the model?

[zeevox]

Taking Qwen2.5-VL as an example, it is trained to output "Qwen HTML" format, with bounding box coordinates added to each HTML tag. There is an example on page 7 of the technical report, and an example cookbook in their repository.

[MyButtermilk]

This model is stronger: https://github.com/PaddlePaddle/PaddleOCR

[jbarlow83]

Paddle is coming soon 🙂

[vancleve]

olmocr 2 looks good too: https://allenai.org/blog/olmocr-2