Skip to content

[SPARK-56540][INFRA][3.5] Add libuv1-dev to CI Docker images to fix R package fs build failure#55757

Closed
aajisaka wants to merge 1 commit into
apache:branch-3.5from
aajisaka:spark-56540-branch-3.5
Closed

[SPARK-56540][INFRA][3.5] Add libuv1-dev to CI Docker images to fix R package fs build failure#55757
aajisaka wants to merge 1 commit into
apache:branch-3.5from
aajisaka:spark-56540-branch-3.5

Conversation

@aajisaka
Copy link
Copy Markdown
Member

@aajisaka aajisaka commented May 8, 2026

What changes were proposed in this pull request?

Add libuv1-dev to the apt-get install list in all two CI Dockerfiles that install R packages:

  • dev/infra/Dockerfile
  • dev/create-release/spark-rm/Dockerfile.base

Why are the changes needed?

The R package fs now requires system library libuv (uv.h). fs is a transitive dependency of both rmarkdown (via sassbslib) and testthat (via pkgload).

When the Docker layer cache is valid, the pre-built R packages are reused and everything works fine. But when the cache is invalidated (e.g. base image digest changes), the R packages are compiled from source. Without libuv1-dev installed, fs fails to configure:

Configuration failed because libuv was not found. Try installing:
 * deb: libuv1-dev (Debian, Ubuntu, etc)
...
<stdin>:1:10: fatal error: uv.h: No such file or directory
ERROR: configuration failed for package 'fs'

This triggers a cascade:

fs (missing libuv) → FAIL
  └→ sass → FAIL
       └→ bslib → FAIL
            └→ rmarkdown → FAIL
  └→ pkgload → FAIL
       └→ testthat → FAIL

The image build itself still "succeeds" because install.packages() in R does not return a non-zero exit code when individual packages fail to install -- it logs warnings but continues. So the Docker image gets pushed with rmarkdown and testthat missing, and the SparkR job fails at test time:

Error in library(testthat) : there is no package called 'testthat'
Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics:
there is no package called 'rmarkdown'

Example failure in branch-3.5 with several fixes (#55753): https://github.com/aajisaka/spark/actions/runs/25539393002/job/74963287548

Does this PR introduce any user-facing change?

No

How was this patch tested?

Verified the fix in a separate PR #55764. This is because now this backport PR itself cannot fix the build error. Now we need to apply both this PR and #55753.

Was this patch authored or co-authored using generative AI tooling?

No

@aajisaka aajisaka changed the title [SPARK-56540][INFRA] Add libuv1-dev to CI Docker images to fix R package fs build failure [SPARK-56540][INFRA][3.5] Add libuv1-dev to CI Docker images to fix R package fs build failure May 8, 2026
@aajisaka
Copy link
Copy Markdown
Member Author

Verified rmarkdown is installed successfully with this patch: https://github.com/aajisaka/spark/actions/runs/25553067367/job/75005477935#step:7:2714

#25 309.9 * installing *source* package 'rmarkdown' ...
#25 309.9 ** this is package 'rmarkdown' version '2.31'
#25 309.9 ** package 'rmarkdown' successfully unpacked and MD5 sums checked
#25 309.9 ** using staged installation
#25 310.0 ** R
#25 310.0 ** inst
#25 310.0 ** byte-compile and prepare package for lazy loading
#25 312.7 ** help
#25 312.9 *** installing help indices
#25 313.0 *** copying figures
#25 313.0 ** building package indices
#25 313.3 ** installing vignettes
#25 313.3 ** testing if installed package can be loaded from temporary location
#25 313.6 ** testing if installed package can be loaded from final location
#25 314.0 ** testing if installed package keeps a record of temporary installation path
#25 314.0 * DONE (rmarkdown)

@aajisaka
Copy link
Copy Markdown
Member Author

Hi @LuciferYang would you review this?

@LuciferYang
Copy link
Copy Markdown
Contributor

#55432 I think @holdenk is currently investigating a more thorough solution to fix the image build issue on branch-3.5.

@aajisaka
Copy link
Copy Markdown
Member Author

Thank you. That's good to know. Here's my progress just FYI: #55764

Now the remaining problem is R version difference. The image now installs R 4.5.0, while the branch-3.5 build cache has R 4.3.3. The old R version doesn't exist in the repository, so it's not easy to install. The version difference causes several build and test failures.

@aajisaka
Copy link
Copy Markdown
Member Author

Closing this as duplicate.

@aajisaka aajisaka closed this May 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants