[Question]: difference between command line and ocrmypdf.ocr() API

[cfoq]

Describe the bug

I have a file that is a PDF consisting of an SVG of dummy text (attached). If I run from the command line:
python -m ocrmypdf dummy.pdf dummyocr.pdf
The file dummyocr.pdf is generated as expected with actual text that can be selected.

If I run the following code in a python file as described in the docs:

import ocrmypdf
from ocrmypdf import OcrOptions
if __name__ == '__main__':
    
    options = OcrOptions(
        input_file="dummy.pdf",
        output_file="dummyocr2.pdf",
    )

    ocrmypdf.ocr(options)

The outpuf file dummyocr2.pdf still appears to have an SVG and the text is NOT selectable.

Here is the output from the command line:

Scanning contents    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 0:00:00
    1 [tesseract] lots of diacritics - possibly poor OCR                                                                                    tesseract.py:295
OCR                  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 0:00:00
Parsing 1 pages with HocrParser                                                                                                                _graft.py:334
Postprocessing...                                                                                                                                 ocr.py:156
[WinError 2] The system cannot find the file specified                                                                                        _windows.py:87
Auto mode: no verapdf available and input is not PDF/A, outputting PDF                                                                     _pipeline.py:1078
Linearizing          ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 100/100 0:00:00
Recompressing JPEGs  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
Deflating JPEGs      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
JBIG2                ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
[WinError 2] The system cannot find the file specified                                                                                        _windows.py:87
[WinError 2] The system cannot find the file specified                                                                                        _windows.py:87
Image optimization ratio: 1.00 savings: 0.0%                                                                                               _pipeline.py:1175
Total file size ratio: 0.93 savings: -7.7%                                                                                                 _pipeline.py:1178
Output file is a PDF (auto mode)            

And here is the output from the API call:

Scanning contents    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 0:00:00
OCR                  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 0:00:00
[WinError 2] The system cannot find the file specified
Linearizing          ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 100/100 0:00:00
Recompressing JPEGs  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
Deflating JPEGs      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
JBIG2                ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0/0 -:--:--
[WinError 2] The system cannot find the file specified
[WinError 2] The system cannot find the file specified

Notably, the command line indicates "Parsing 1 pages with HocrParser" whereas the API call does not. Do I need to specify different params to trigger that? Or any other reason these are not working the same?

I have tried various combinations of options in the API call such as force_ocr=True, redo_ocr=True (not concurrently with force_ocr), languages=['eng',], output_type="pdf", but cannot reproduce what is done with the command line.

Note, I believe the error "The system cannot find the file specified" is because GhostScript is NOT installed.

Any help appreciated. Thank you!

Steps to reproduce

See description

Files

dummy.pdf
dummyocr.pdf
dummyocr2.pdf

How did you download and install the software?

PyPI (pip, poetry, pipx, etc.)

OCRmyPDF version

17.2.0

Relevant log output

see output in description

[cfoq]

I was able to replicate the command line behavior with the following code. Is this a reasonable approach or any other suggestions?

from ocrmypdf.api import run_pipeline
from ocrmypdf.cli import get_options_and_plugins

options, plugins = get_options_and_plugins(args=[input_pdf, output_pdf])
run_pipeline(options=options, plugin_manager=plugins)

[tmeckel]

@cfoq ran into the same issue. You have to set a tesseract timeout when you use the API. The CLI uses an implicit timeout setting that the API does not have. tesseract will stop immediately and thus there is no text-overlay in the final output file. Took me half a day to figure this out:

https://github.com/ophiosdev/ocrpdf-container/blob/master/app/worker.py#L137

[cfoq]

@tmeckel Thank you! I was tracing through the code but had not found that yet. Looks like the default for ocr_job_timeout is 3600, along with a bunch of other default values in settings.Config.

https://github.com/ophiosdev/ocrpdf-container/blob/0a0813d96a9e8f15bed7e2f8315be776c1cfe9d6/app/settings.py#L69

So, back to my other question (hopefully one of the authors can set me straight): should we call ocrmypdf.ocr(...) and set all of those options? Or use the run_pipeline(...) which seems to work as intended?

[tmeckel]

https://github.com/ophiosdev/ocrpdf-container/blob/0a0813d96a9e8f15bed7e2f8315be776c1cfe9d6/app/settings.py#L69

@cfoq don't confuse the configuration of the "ocrpdf-container" with the configuration that is used to call ocrpdf via it's API.

https://github.com/ophiosdev/ocrpdf-container/blob/master/app/worker.py#L123C1-L139C10

And I think you'll be good with adding tesseract_timeout and tesseract_non_ocr_timeout to your test configuration like so:

import ocrmypdf
from ocrmypdf import OcrOptions
if __name__ == '__main__':
    
    options = OcrOptions(
        input_file="dummy.pdf",
        output_file="dummyocr2.pdf",
        tesseract_timeout=3600,
        tesseract_non_ocr_timeout=3600,
    )

    ocrmypdf.ocr(options)

[cfoq]

@tmeckel I tried your suggestion and it worked. Thank you for the help and clarification!

@jbarlow83 Would it be possible to update the documentation? The page on using the API could be clarified to specify the Tesseract settings when calling the .ocr() method to replicate the default behavior of running from the command line.

[tmeckel]

@cfoq @jbarlow83 I'd opt for meaningful defaults on the OcrOptions class as well and of course some polishing on the API documentation.