RSDK-14187: add parallel Focal static-build image#6206
Conversation
Add a self-contained Ubuntu Focal (20.04) image that builds RDK's shipped
semi-static viam-server, in parallel with the existing external
canon/rdk-devenv/antique2 images (which keep building untouched). Focal is
the oldest distro with a usable C++20 libstdc++ (GCC 10), the minimum needed
to consume trajex's C++20 cgo bindings while staying compatible with Jetpack 5.
etc/Dockerfile.focal (multi-stage: base -> cache-builder -> final, mirroring
Dockerfile.antique-cache so the -cache image ships a warmed Go module cache):
- gcc-10/g++-10 (C++20 floor), Go pinned to go.mod, symlinked onto sudo's
secure_path, testbot uid/gid 1000
- from-source deps matched to the antique container of record: nlopt 2.7.1,
cmake 3.26.2 (Kitware), upx 4.2.2, plus libjpeg-turbo8-dev
- x264 built from source: Focal's apt libx264.a references __*_finite libm
symbols removed in glibc 2.31, so it is unlinkable for the static build
- nlopt 2.7.1 from source: Focal's apt nlopt 2.6.1 lacks nlopt_set_param /
nlopt_num_params required by the go-nlopt binding, and ships no static archive
- drops antique cruft no longer needed: git-from-source, tflite, pigpio,
node, license_finder
docker.yml: publish ghcr.io/viamrobotics/rdk-focal:{amd64,arm64,armhf}-cache
alongside the existing images.
focal-build.yml: non-gating validation (schedule / dispatch / focal-build
label) that builds viam-server-static (invoked like antique, sudo -Hu testbot),
checks its GLIBC/GLIBCXX floor, boot-smoke-tests it, and uploads
viam-server-focal-<arch> to packages.viam.com/.../testing/focal/<date>/<sha>/
for on-device testing.
setup.sh: soften the native-Focal warning now that Focal is the baseline.
Verified locally on arm64: viam-server-static links with only libc-family
dynamic deps (no libx264/libnlopt/libstdc++), a GLIBC_2.29 floor, and boots.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
9b2814b to
bb6d714
Compare
Make focal-build.yml a reusable workflow that builds via `make BUILD_CHANNEL=<channel> static-release`, verifies the GLIBC/GLIBCXX floor, and boot-smoke-tests the artifact. Scheduled runs validate only; workflow_dispatch publishes the `focal` channel. pullrequest-trusted.yml gains a focal job (gated on `safe to test` + `focal-build`) that calls focal-build.yml with release_type pr, publishing a namespaced `focal-pr-<number>` channel so PR builds never override the shared focal channel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Building on Focal already caps the GLIBC floor at 2.31, so the objdump check could not fail; the boot smoke test on the Focal image is the real proof of runnability. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| @@ -0,0 +1,118 @@ | |||
| name: Focal Static Build | |||
There was a problem hiding this comment.
when we're directionally in agreement, i'd like to merge this PR so i can follow up with improvements on a new branch and run the workflow from the branch
There was a problem hiding this comment.
I'm happy for you to merge it, for my part.
| cmake --version | ||
|
|
||
| # nlopt into /usr/local: headers + PIC static lib + pkg-config. | ||
| RUN set -eux; \ |
There was a problem hiding this comment.
Would it make sense to extract all of these run commands to a single script? Seems like its mostly a single concern: ensuring that the appropriate versions of the packages are installed, something like the ensure-dependencies scripts we use elsewhere.
|
|
||
| # Final image: base plus the warmed module cache. | ||
| FROM base | ||
| COPY --from=cache-builder --chown=1000:1000 /home/testbot/go /home/testbot/go |
There was a problem hiding this comment.
This is intended to capture the Go build cache, correct? Should we explicitly set GOCACHE to ensure that its always where we think it is?
There was a problem hiding this comment.
I don't have a great deal of insight into go caching, but I wonder if this belongs in the container, or in the github action setup. Maybe we can kick it out for now in the interest of not needing to answer that question now and then come back to it later?
There was a problem hiding this comment.
i'm fine with kicking off caching, i don't think this workflow needs to be perfect from day 1
There was a problem hiding this comment.
Yeah I think that makes sense. I think this might be a useful resource down the line if we want to speed up CI process, but not a necessity at this stage.
- docker.yml: fail-fast:false so a failing rdk-focal leg can't cancel the rdk-devenv/antique2 cache rebuilds - focal-build.yml: drop concurrency block; github.ref is the base ref for every PR via workflow_call, so the shared group cancelled unrelated builds - Dockerfile.focal: keep the final image root-default like antique2 (confirmed via image config); scope USER testbot to cache-builder Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- drop -f from the smoke-test curl (match staticbuild: any HTTP response means the server is up) - workflow_dispatch now takes release_type (default: validate), so a manual run only publishes the shared focal channel when explicitly asked; publish steps gate on one resolved flag Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…real version Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| && update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100 \ | ||
| --slave /usr/bin/g++ g++ /usr/bin/g++-10 \ |
There was a problem hiding this comment.
I'm not convinced we need this. The default system GCC, at GCC 9, should be sufficient to consume the C ABI stuff. It is only when building a C++20 target that we need GCC 10.
The system C++ runtime is already at a GCC 10 level. Yes this is a little odd/surprising, but quite true if you examine the packages.
Anyway, I think you can drop it, which makes it one less thing to do.
| # Build/link deps. gcc-10 sets the C++20 floor; nasm builds x264. | ||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| build-essential \ | ||
| gcc-10 g++-10 \ |
There was a problem hiding this comment.
See below, I don't think you need this.
| # upx, for the -compressed release targets. | ||
| RUN set -eux; \ | ||
| case "$(dpkg --print-architecture)" in \ | ||
| amd64) upx_arch=amd64 ;; \ | ||
| arm64) upx_arch=arm64 ;; \ | ||
| armhf) upx_arch=arm ;; \ | ||
| *) echo "unsupported arch" >&2; exit 1 ;; \ | ||
| esac; \ | ||
| curl -fsSL "https://github.com/upx/upx/releases/download/v${UPX_VERSION}/upx-${UPX_VERSION}-${upx_arch}_linux.tar.xz" \ | ||
| | tar -C /usr/local/bin/ --strip-components=1 --wildcards -xJ '*/upx' |
There was a problem hiding this comment.
See my comment re UPX in https://viam.atlassian.net/browse/RSDK-14187, this doesn't belong in the container. UPX is architecture independent, and it should be run as a release workflow level artifact processing step. It also doesn't belong in the Makefile, for that matter.
|
|
||
| # Final image: base plus the warmed module cache. | ||
| FROM base | ||
| COPY --from=cache-builder --chown=1000:1000 /home/testbot/go /home/testbot/go |
There was a problem hiding this comment.
I don't have a great deal of insight into go caching, but I wonder if this belongs in the container, or in the github action setup. Maybe we can kick it out for now in the interest of not needing to answer that question now and then come back to it later?
| # GO_VERSION tracks the `go` directive in go.mod. | ||
| ARG GO_VERSION=1.25.9 | ||
| ARG NLOPT_VERSION=2.7.1 | ||
| ARG CMAKE_VERSION=3.26.2 |
There was a problem hiding this comment.
- Cmake is up to 4 something now. Upgrade?
- nlopt is up to 2.11 now. Upgrade?
The go:embed needing control.js was removed with the local control page (aa16e81); the server builds without it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- drop gcc-10, system gcc 9 suffices - nlopt 2.11.0 built C-only (NLOPT_CXX=OFF); cmake 4.3.4, pip wheel on armhf - move upx from image to workflow step - drop Go cache-builder stage - keep x264 source build: focal's prebuilt libx264.a fails static link (undefined __log2f_finite on arm64) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ocal # Conflicts: # .github/workflows/pullrequest-trusted.yml
|
Andrew C. Morrow (@acmorrow) fyi i addressed all other comments but still looking on x264. |
Andrew C. Morrow (acmorrow)
left a comment
There was a problem hiding this comment.
The docker container looks good to me. I did some testing with it and I was able to build against trajex in it.
I don't have as much to say about the focal-build.yml file, as I'm not particularly familiar with the RDK build/channel/publish/release flow. I think if Alexander Maschas (@amaschas) wants to share his thoughts on it we could probably get this all merged.
| - name: Fix permissions | ||
| run: chown -R testbot:testbot . | ||
|
|
||
| - name: Install upx |
There was a problem hiding this comment.
I think there is a github actions marketplace implementation of this that might be worth investigating if you are willing to accept the external dep.
There was a problem hiding this comment.
i'll follow up with another pr. i can't really test the flow until i merge so it would be nice to see it work
There was a problem hiding this comment.
Sounds good, I'm just happy to see it out of the container!
Follow-up from #6206 review: the RUN blocks are one concern (install pinned toolchain versions), so move them to etc/ensure-focal-deps.sh. Versions live in the script since nothing overrides them as build args. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Follow-up from #6206 review: use the marketplace action instead of a hand-rolled curl install. The action installs into the runner tool cache and only prepends it to the step PATH, so symlink upx into /usr/local/bin where sudo's secure_path resolves it for the 'sudo -Hu testbot' build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RSDK-14187
Adds a self-contained Ubuntu Focal (20.04) image that builds RDK's semi-static
viam-server, in parallel with the existingcanon/rdk-devenv/antique2images (unchanged). Focal is the oldest distro with a C++20 libstdc++ (GCC 10), required for trajex's cgo bindings and still compatible with Jetpack 5. No cutover here — validation only.Decisions
rdk-focalimage published alongside existing ones; no existing release path changed.libjpeg-turbo8-dev.libx264.areferences__*_finitelibm symbols removed in glibc 2.31, so it can't be static-linked (antique's older glibc still has them).nlopt_set_param/nlopt_num_paramsrequired by go-nlopt (added in 2.7.0) and ships no static archive.gosymlinked into/usr/local/binso it resolves under sudo'ssecure_path(thesudo -Hu testbotbuild path).Dockerfile.antique-cache(multi-stage;-cacheimage ships a warmed Go module cache).focalrelease channel viafocal-build.yml(reusable workflow,make BUILD_CHANNEL=<channel> static-release): scheduled runs validate only;workflow_dispatchpublishesviam-server-focal-<arch>to the release root. Labeled PRs go throughpullrequest-trusted.yml(gated onsafe to test+focal-build) and publish a namespacedviam-server-focal-pr-<number>-<arch>, so PR builds never override the sharedfocalchannel — same model asstatic-build.Verification (local, arm64)
viam-server-staticlinks with only libc-family dynamic deps (no libx264/libnlopt/libstdc++), GLIBC_2.29 floor, and boots.sudo -Hu testbot make server-staticresolvesgoand warms a 1.5 GB module cache. The full in-Docker-cachebuild wasn't run locally (flakyproxy.golang.orgegress); it runs in CI viadocker.yml.Out of scope (deferred)
Cutover off antique2/rdk-devenv; pinning old tags by digest;
ld_wrapper.shde-static of libstdc++/libgcc;full-staticCGO_ENABLED=0fix; moving upx to CI;-DNLOPT_CXX=OFF; dev-env Focal image. No viam-agent OS gate exists, so nothing needed there.🤖 Generated with Claude Code