[Bug]: RHEL 9 requires ghostscript 9.54 to work

[nisbet-hubbard]

What were you trying to do?

Although RHEL backports things to its gs binary, the version is frozen at 9.54. Which means to use the latest ocrmypdf, one has to get a snap-distributed later gs. But this introduces permissions issues as shown in the log, and because gs options aren’t exposed, there’s no way to get round this.

Currently, we can only get your great ocrmypdf to work on RHEL 9 (the latest) by using v14.4.

Would it be possible by any chance to lower ocrmypdf‘s gs dependency to 9.54 so projects running enterprise linux can benefit from the newest version?

Where are you installing from?

PyPI (pip, poetry, pipx, etc.)

What operating system are you working on?

Linux

Relevant log output

Start processing 2 pages concurrently                                                                              ocr.py:98
    1 Error: /undefinedfilename in (/tmp/ocrmypdf.io.9d25egvn/origin.pdf)                                 ghostscript.py:139
Operand stack:                                                                                                              
Execution stack:   %interp_exit   .runexec2   --nostringval--   --nostringval--   --nostringval--   2                       
%stopped_push   --nostringval--   --nostringval--   --nostringval--   false   1   %stopped_push                             
Dictionary stack:   --dict:751/1123(ro)(G)--   --dict:0/20(G)--   --dict:85/200(L)--                                        
Current allocation mode is local                                                                                            
Last OS error: No such file or directoryGPL Ghostscript 10.02.1: Unrecoverable error, exit code 1

[jbarlow83]

For ocrmypdf 15, I decided to make Ubuntu 22.04 the baseline system mainly because that is what GitHub's CI provides. This is where the ghostscript 9.55 requirement comes from. It's very likely 9.54 would work well for most users - I'm not aware of any technical reason why it wouldn't.

It would be best to confirm RHEL9 has backported the fix for CVE-2023-43115 to Ghostscript 9.54. The patch looks trivial so this has probably been done already. I'd appreciate check this and also confirm that ocrmypdf's test suite passes on your RHEL9 environment when the ghostscript version check in src/ocrmypdf/builtin_plugins/ghostscript.py line 57 patched to 9.54 instead.

Assuming the above are both true, I can modify the minimum ghostscript version to 9.54 and introduce a maintainer-friendly variable to silence the Ghostscript CVE warning message. Then you'll have a native install with a one-line patch to suppress the warning, without the complications of snap.

---

If we want to go the snap route, the issue is that the Ghostscript snap and ocrmypdf need access to a common TMPDIR. If this were Docker we'd bind-mount ocrmypdf's /tmp/ocrmypdf.io.whatever folder to the container with -v {ocrmypdf_tmp}:/tmp. Not sure how to achieve the same with snap, but that's the general idea.

[nisbet-hubbard]

Thank you so much for providing a clear roadmap forward.

I can confirm the CVE has been patched for 9.54: see https://access.redhat.com/errata/RHSA-2023:6732 and toggle to ‘Updated Packages’.

I agree it’s preferable to avoid the complications of snap. I’ll report back after compiling the patched code.

[nisbet-hubbard]

I append the test log for main:

---------------------------------------------------- Captured log call -----------------------------------------------------
WARNING  ocrmypdf.builtin_plugins.ghostscript:ghostscript.py:66 The installed version of Ghostscript 9.54.0, contains a remote code execution security vulnerability. Please upgrade to a newer version. For details see CVE-2023-43115. The issue is not known to affect OCRmyPDF or processing PDFs with Ghostscript, but upgrading Ghostscript is recommended.
ERROR    ocrmypdf._exec.tesseract:tesseract.py:252 [tesseract] Error opening data file /usr/share/tesseract/tessdata/osd.traineddata
===================================================== warnings summary =====================================================
tests/test_main.py: 21 warnings
tests/test_metadata.py: 4 warnings
tests/test_preprocessing.py: 9 warnings
tests/test_rotation.py: 4 warnings
tests/test_tagged.py: 1 warning
tests/test_tesseract.py: 1 warning
  /root/.venv/lib64/python3.11/site-packages/pikepdf/_methods.py:266: UserWarning: pikepdf is missing some specialized decoders (probably JBIG2) so not all stream contents can be tested.
    self._decode_all_streams_and_discard()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================= short test summary info ==================================================
FAILED tests/test_helpers.py::TestFileIsWritable::test_chmod - AssertionError: assert not True
FAILED tests/test_hocrtransform.py::test_mono_image - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pytest-2/popen-gw0/test_mono_image0/blank....
FAILED tests/test_main.py::test_pagesize_consistency[hocr] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/ocrmypdf.io.wkuvfbmk/000001_ocr_hocr.hocr'
FAILED tests/test_pdfa.py::test_pdfa[1-3] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pytest-2/popen-gw0/test_pdfa_1_3_0/out.pdf'
FAILED tests/test_pdfa.py::test_pdfa[2-3] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pytest-2/popen-gw0/test_pdfa_2_3_0/out.pdf'
FAILED tests/test_pdfa.py::test_pdfa[3-3] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pytest-2/popen-gw0/test_pdfa_3_3_0/out.pdf'
FAILED tests/test_main.py::test_input_file_not_readable - assert <ExitCode.ok: 0> == <ExitCode.input_file: 2>
FAILED tests/test_optimize.py::test_jbig2_lossy[False] - ocrmypdf.exceptions.MissingDependencyError: Could not find program 'pngquant' on the PATH
FAILED tests/test_optimize.py::test_jbig2_lossy[True] - ocrmypdf.exceptions.MissingDependencyError: Could not find program 'pngquant' on the PATH
FAILED tests/test_rotation.py::test_rotate_deskew_ocr_timeout - ocrmypdf.exceptions.SubprocessOutputError
====================== 10 failed, 307 passed, 45 skipped, 4 xfailed, 40 warnings in 346.82s (0:05:46) ======================

For comparison, here’s the results for v14.4.0 which we’ve been using:

FAILED tests/test_main.py::test_quick - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/
ccitt/__-l__eng__000001_ocr.png__00...
FAILED tests/test_main.py::test_oversample[hocr] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/t
ests/cache/skew/__-l__eng__000001_ocr.png__000...
FAILED tests/test_main.py::test_oversample[sandwich] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyP
DF/tests/cache/skew/__-l__eng__000001_ocr.png__000...
FAILED tests/test_helpers.py::TestFileIsWritable::test_chmod - AssertionError: assert not True
FAILED tests/test_main.py::test_jbig2_passthrough - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/
tests/cache/jbig2/__-l__eng__000001_ocr.png__00...
FAILED tests/test_main.py::test_maximum_options[pdf-hocr] - FileNotFoundError: [Errno 2] No such file or directory: '/root/O
CRmyPDF/tests/cache/multipage/__-l__eng__--psm__2__0000...
FAILED tests/test_main.py::test_pagesize_consistency[hocr] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/o
crmypdf.io.q24_qzaz/000001_ocr_hocr.hocr'
FAILED tests/test_main.py::test_maximum_options[pdf-sandwich] - FileNotFoundError: [Errno 2] No such file or directory: '/ro
ot/OCRmyPDF/tests/cache/multipage/__-l__eng__--psm__2__0000... 
FAILED tests/test_main.py::test_maximum_options[pdfa-hocr] - FileNotFoundError: [Errno 2] No such file or directory: '/root/
OCRmyPDF/tests/cache/multipage/__-l__eng__--psm__2__0000...
FAILED tests/test_main.py::test_maximum_options[pdfa-sandwich] - FileNotFoundError: [Errno 2] No such file or directory: '/r
oot/OCRmyPDF/tests/cache/multipage/__-l__eng__--psm__2__0000...
FAILED tests/test_main.py::test_sidecar_nonempty - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/t
ests/cache/ccitt/__-l__eng__000001_ocr.png__00...
FAILED tests/test_main.py::test_pdfa_n[1] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/ca
che/ccitt/__-l__eng__000001_ocr.png__00...
FAILED tests/test_main.py::test_pdfa_n[2] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/ca
che/ccitt/__-l__eng__000001_ocr.png__00...
FAILED tests/test_main.py::test_pdfa_n[3] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/ca
che/ccitt/__-l__eng__000001_ocr.png__00...
FAILED tests/test_main.py::test_input_file_not_readable - assert <ExitCode.ok: 0> == <ExitCode.input_file: 2>
FAILED tests/test_main.py::test_pagesegmode[hocr] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/
tests/cache/skew/__-l__eng__--psm__7__000001_oc...
FAILED tests/test_main.py::test_pagesegmode[sandwich] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmy
PDF/tests/cache/skew/__-l__eng__--psm__7__000001_oc...
FAILED tests/test_optimize.py::test_jbig2_lossy[False] - ocrmypdf.exceptions.MissingDependencyError: Could not find program 
'pngquant' on the PATH
FAILED tests/test_optimize.py::test_jbig2_lossy[True] - ocrmypdf.exceptions.MissingDependencyError: Could not find program '
pngquant' on the PATH
FAILED tests/test_page_numbers.py::test_limited_pages - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmy
PDF/tests/cache/multipage/__-l__eng__000005_ocr.png...
FAILED tests/test_pdfa.py::test_pdfa[1-3] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pyt
est-4/popen-gw1/test_pdfa_1_3_0/out.pdf'
FAILED tests/test_pdfa.py::test_pdfa[2-3] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pyt
est-4/popen-gw1/test_pdfa_2_3_0/out.pdf'
FAILED tests/test_pdfa.py::test_pdfa[3-3] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pyt
est-4/popen-gw1/test_pdfa_3_3_0/out.pdf'
FAILED tests/test_preprocessing.py::test_exotic_image[pdfa-sandwich-palette.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/palette/__-l__eng__--psm__2__000001...
FAILED tests/test_preprocessing.py::test_exotic_image[pdfa-sandwich-ccitt.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/ccitt/__-l__eng__000001_ocr.png__00...
FAILED tests/test_preprocessing.py::test_exotic_image[pdfa-sandwich-jbig2.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/jbig2/__-l__eng__--psm__2__000001_r...
FAILED tests/test_preprocessing.py::test_exotic_image[pdfa-sandwich-lichtenstein.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/lichtenstein/__-l__eng__000001_ocr....
FAILED tests/test_preprocessing.py::test_exotic_image[pdfa-hocr-palette.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/palette/__-l__eng__--psm__2__000001...
FAILED tests/test_preprocessing.py::test_exotic_image[pdfa-hocr-ccitt.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/ccitt/__-l__eng__000001_ocr.png__00...
FAILED tests/test_preprocessing.py::test_exotic_image[pdfa-hocr-jbig2.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/jbig2/__-l__eng__--psm__2__000001_r...
FAILED tests/test_preprocessing.py::test_exotic_image[pdfa-hocr-lichtenstein.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/lichtenstein/__-l__eng__000001_ocr....
FAILED tests/test_preprocessing.py::test_non_square_resolution[hocr] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/aspect/__-l__eng__000001_ocr.png__0...
FAILED tests/test_preprocessing.py::test_non_square_resolution[sandwich] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/aspect/__-l__eng__000001_ocr.png__0...
FAILED tests/test_preprocessing.py::test_convert_to_square_resolution[hocr] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/aspect/__-l__eng__000001_ocr.png__0...
FAILED tests/test_preprocessing.py::test_convert_to_square_resolution[sandwich] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/aspect/__-l__eng__000001_ocr.png__0...
FAILED tests/test_rotation.py::test_autorotate_threshold[1-ge-0.95] - ocrmypdf.exceptions.SubprocessOutputError
FAILED tests/test_rotation.py::test_autorotate_threshold[99-le-0.9] - ocrmypdf.exceptions.SubprocessOutputError
FAILED tests/test_rotation.py::test_rotate_deskew_ocr_timeout - ocrmypdf.exceptions.SubprocessOutputError
FAILED tests/test_preprocessing.py::test_exotic_image[pdf-sandwich-palette.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/palette/__-l__eng__--psm__2__000001...
FAILED tests/test_preprocessing.py::test_exotic_image[pdf-sandwich-ccitt.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/ccitt/__-l__eng__000001_ocr.png__00...
FAILED tests/test_preprocessing.py::test_exotic_image[pdf-sandwich-jbig2.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/jbig2/__-l__eng__--psm__2__000001_r...
FAILED tests/test_preprocessing.py::test_exotic_image[pdf-sandwich-lichtenstein.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/lichtenstein/__-l__eng__000001_ocr....
FAILED tests/test_preprocessing.py::test_exotic_image[pdf-hocr-palette.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/palette/__-l__eng__--psm__2__000001...
FAILED tests/test_preprocessing.py::test_exotic_image[pdf-hocr-ccitt.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/ccitt/__-l__eng__000001_ocr.png__00...
FAILED tests/test_preprocessing.py::test_exotic_image[pdf-hocr-jbig2.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/jbig2/__-l__eng__--psm__2__000001_r...
FAILED tests/test_preprocessing.py::test_exotic_image[pdf-hocr-lichtenstein.pdf] - FileNotFoundError: [Errno 2] No such file or directory: '/root/OCRmyPDF/tests/cache/lichtenstein/__-l__eng__000001_ocr....
FAILED tests/test_userunit.py::test_rotate_interaction - ocrmypdf.exceptions.SubprocessOutputError
===================== 47 failed, 255 passed, 45 skipped, 4 xfailed, 1593 warnings in 150.20s (0:02:30) =====================

[jbarlow83]

Doesn't look that bad actually... I think this is all configuration.

Did you base the packaging scripts on Fedora's or this being done anew? Whatever Fedora does/did works.

Probably fixes for each log line below:

ERROR    ocrmypdf._exec.tesseract:tesseract.py:252 [tesseract] Error opening data file /usr/share/tesseract/tessdata/osd.traineddata

^ Need to install tesseract-osd (mandatory dependency).

  /root/.venv/lib64/python3.11/site-packages/pikepdf/_methods.py:266: UserWarning: pikepdf is missing some specialized decoders (probably JBIG2) so not all stream contents can be tested.
    self._decode_all_streams_and_discard()

^ Need to install jbig2dec (optional but recommended).

FAILED tests/test_helpers.py::TestFileIsWritable::test_chmod - AssertionError: assert not True
FAILED tests/test_hocrtransform.py::test_mono_image - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pytest-2/popen-gw0/test_mono_image0/blank....
FAILED tests/test_main.py::test_pagesize_consistency[hocr] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/ocrmypdf.io.wkuvfbmk/000001_ocr_hocr.hocr'
FAILED tests/test_pdfa.py::test_pdfa[1-3] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pytest-2/popen-gw0/test_pdfa_1_3_0/out.pdf'
FAILED tests/test_pdfa.py::test_pdfa[2-3] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pytest-2/popen-gw0/test_pdfa_2_3_0/out.pdf'
FAILED tests/test_pdfa.py::test_pdfa[3-3] - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-root/pytest-2/popen-gw0/test_pdfa_3_3_0/out.pdf'
FAILED tests/test_main.py::test_input_file_not_readable - assert <ExitCode.ok: 0> == <ExitCode.input_file: 2>

^ Looks as if /tmp/pytest-of-root folder is not writable by the user running the test. You could also set the TMPDIR environment variable to a writable location for testing.

FAILED tests/test_optimize.py::test_jbig2_lossy[False] - ocrmypdf.exceptions.MissingDependencyError: Could not find program 'pngquant' on the PATH
FAILED tests/test_optimize.py::test_jbig2_lossy[True] - ocrmypdf.exceptions.MissingDependencyError: Could not find program 'pngquant' on the PATH

^ Need to install pngquant (required for test suite, optional but recommended for runtime).

FAILED tests/test_rotation.py::test_rotate_deskew_ocr_timeout - ocrmypdf.exceptions.SubprocessOutputError

^ Likely due to tesseract-osd not being installed.

[nisbet-hubbard]

Nice to have your instructions on how to deal with each.

I wasn’t using any packaging script, in part because most of the tesseract- dependencies on Fedora haven’t been branched for RHEL/EPEL. I was able to manually add tessdata/ files, as well as installing jbig2de and pngquant.

Since it looks like this has all been sorted out, would it be possible to go ahead and modify the minimum ghostscript version to 9.54?

[nisbet-hubbard]

Completed in v16.0.3.

[nemobis]

Last OS error: No such file or directoryGPL Ghostscript 10.02.1: Unrecoverable error, exit code 1

I get this error in Fedora 41 with ocrmypdf 16.7.0 and gs 10.03.1, even installing from fedora repositories. (Same if I downgrade to ocrmypdf-0:16.4.3-1.fc41.)