[Feature]: `nbocr` has out-of-the-box Metal, OpenCL, Vulkan and CUDA!

[SuperCowProducts]

Describe the proposed feature

My machine's best GPU acceleration (like many others, especially mobile devices now) is Vulkan, so nbocr is pretty exciting (the default models are very fast, I'd say comparable to Tesseract (an objective measure of just how much better it may be would be interesting, not sure how to go about that though).

Only obstacle is JSON->PDF embeddings, any suggestions for this?

I did find a very good Paddle-OCR substitute (excellent LaTeX support and everything, which the faster nbocr is missing) which worked well even just on CPU (but should be possible to get GPU working once PyTorch is compiled for that GPU (e.g. OpenCL, Vulkan), but I got stuck on this same step...

[SuperCowProducts]

Hmm, so I think the most straightforward path is building some kind of conversion algorithm that takes JSON (from whatever OCR program it may come from) and reformats it into hOCR (I can see off the bat that hocr-pdf from hocr-tools by ocropus does this and I will start with it, although searching the web other similar-looking programs appear, so may be worth checking out those as well if one feels the need to.

The JSON may vary slightly from program to program, but in the end the type of data stored is the same, so once it works with say nbocr, some simple tweaks should allow it to work with surya / many others.

[SuperCowProducts]

In fact, the best strategy may be combining different methods: I've just tested surya_gui, which takes 45-60 seconds per page (even if I've built PyTorch with Vulkan, figuring out where the bottleneck is would be interesting: I wasn't able to get GPU percentage usage in Asahi Linux working, but CPU percentage is between 80 and 90% and doesn't max out at 100%, which is a sign GPU is assisting.

I'd say everything about the OCR is pretty stunning though (handwriting, LaTeX conversion, tables...).

Maybe a system that detects when OCR is getting difficult / poor (a bit like when tesseract starts reporting 'possible poor OCR' or something like that, but because it detects handwriting / LaTeX / tables / wonky text etc.) and passes for example from nbocr to surya?