mozjs128: serve modified Source0 (keep only js/-build subtrees)#17208
Draft
PawelWMS wants to merge 1 commit into
Draft
mozjs128: serve modified Source0 (keep only js/-build subtrees)#17208PawelWMS wants to merge 1 commit into
PawelWMS wants to merge 1 commit into
Conversation
PawelWMS
force-pushed
the
pawelwi/mozjs128-strip-source
branch
13 times, most recently
from
b4d1e43 to
0650fc3
Compare
`mozjs128` builds the SpiderMonkey JavaScript engine from the
upstream Firefox ESR source tarball. The full tarball ships a long
tail of artefacts that are never compiled or installed by this
component -- vendored Windows PE binaries (NSIS plugin DLLs, 7-Zip
stubs, telemetry / mozapps / mozbase test fixtures, signed
`msvcp140.dll`), oss-fuzz seed corpora, deliberately-malformed
media / image / font crash-test inputs, encrypted ZIP test fixtures
-- and those subtrees trip the automated package-signing pipeline's
FS-aware deep scanner on the SRPM payload. `%prep`-time deletions
are too late because the scanner inspects Source0 verbatim, before
`%prep` runs.
This commit overrides Source0 with a locally-modified tarball that
contains only the subtrees the SpiderMonkey build actually touches.
Why a Source0 strip rather than removing the component
------------------------------------------------------
A dependency-impact scan turned up reverse dependencies that must
keep building:
* `specs/c/cjs/cjs.spec`
BuildRequires: pkgconfig(mozjs-128) >= %{mozjs128_version}
Requires: mozjs128%{?_isa} >= %{mozjs128_version}
* `specs/c/cinnamon/cinnamon.spec`
BuildRequires: pkgconfig(cjs-1.0) >= %{cjs_version}
Requires: cjs%{?_isa} >= %{cjs_version}
* `base/comps/components.toml` keeps `[components.cjs]`,
`[components.cinnamon]`, and seven `cinnamon-*` packages.
Removing `mozjs128` would therefore break the `cjs` build and the
entire Cinnamon desktop environment. The Source0 strip preserves
the SpiderMonkey artefacts those consumers need (`libmozjs-128.so*`,
`mozjs-128/` headers, `mozjs-128.pc`) while dropping the Firefox-
only subtrees the scanner flags.
Keep-list
---------
Top-level entries inside `firefox-128.11.0/` that survive the strip:
LICENSE, Cargo.toml, Cargo.lock, configure.py, moz.configure, build,
config, intl, js, mfbt, memory, mozglue, python, third_party. Plus
nested strips of `intl/icu` (we build with `--with-system-icu`),
`js/src/fuzz-tests`, `js/src/devtools/automation/variants`,
`js/src/octane`, and `js/src/ctypes/libffi`. Plus two nested restores:
`testing/mozbase` (the full `testing/` directory is too large and
carries fuzzer corpora / crash fixtures the scanner trips on;
`testing/mozbase` is only 12 MB of plain Python and is the canonical
home of the `mozfile` / `mozinfo` / `mozprocess` etc. helpers the
build's `find_program` machinery imports), and the single 4 KB
header `intl/icu/source/common/unicode/uvernum.h` (which
`js/moz.configure`'s `icu_version()` reads to extract
`U_ICU_VERSION_MAJOR_NUM` even with `--with-system-icu`).
`intl/`, `python/`, `configure.py`, and `testing/mozbase/` are kept
because the spec patches and the configure machinery reach into all
of them. Earlier script iterations dropped each in turn and the
build failed:
* `%prep` failed with "No file to patch. Skipping patch." against
`python/mozbuild/mozbuild/backend/recursivemake.py` and
`intl/icu_sources_data.py` (added `python` and `intl` to
`KEEP_TOP`).
* `%build` failed at `js/src/configure` with `python3: can't open
file '.../configure.py'` (added `configure.py` to `KEEP_TOP`).
* `%build` failed inside SpiderMonkey's `configure` with
`ModuleNotFoundError: No module named 'mozfile'` -- the build
machinery's `find_program` (in
`build/moz.configure/util.configure`) does
`@imports(_from="mozfile", _import="which")`. Restored
`testing/mozbase` via a `NESTED_KEEP` restore step.
* `%build` failed inside `js/moz.configure`'s `icu_version()`
with `FileNotFoundError: '.../intl/icu/source/common/unicode/
uvernum.h'` -- the helper opens that single header to extract
`U_ICU_VERSION_MAJOR_NUM` regardless of `--with-system-icu`.
Restored that file via `NESTED_KEEP`.
Changes
-------
1. `base/comps/mozjs128/mozjs128.comp.toml` -- new dedicated
component file with a single
`[[components.mozjs128.source-files]]` block:
* `filename = "firefox-128.11.0esr.source.tar.xz"` matches the
upstream filename so `mozjs128.spec`'s `Source0:` line does not
need to change.
* `hash` + `origin.uri` point at the locally-modified tarball,
served from the lookaside `repo` container under the
`pkgs_modified/` prefix.
* `replace-upstream = true` + `replace-reason = "..."` swap the
same-named upstream entry in the Fedora `sources` manifest in
place (single-step migration; no separate `file-remove` overlay
needed). `azldev`'s render step emits an audit WARN log naming
the override and the from/to SHA-512 pair.
2. `base/comps/mozjs128/modify_source.sh` -- deterministic
strip-and-repack helper. Downloads the upstream
`firefox-128.11.0esr.source.tar.xz`, verifies its SHA-512, deletes
everything outside the SpiderMonkey-build keep list, repacks
deterministically (`tar --sort=name --owner=0 --group=0
--numeric-owner --mtime=... | xz -T 1 -9e`), and prints the
resulting SHA-512 plus a ready-to-paste `az storage blob upload`
command. Output lives under
`<repo-root>/base/build/work/scratch/mozjs128/` (covered by the
top-level `.gitignore` via `build/`).
The script is byte-deterministic: identical upstream input ⇒
byte-identical output ⇒ identical SHA-512 across machines and
re-runs. The pinned modified-tarball SHA-512 is:
a79fe02e82493577e19d08a287415d2bbe94727dabd20cc162bc35c1e37d35
da2eccfee81da50e8abefecadac5510f66cd28cf34466f53cbf23c56bf9020f5bc
3. `base/comps/components.toml` -- inline `[components.mozjs128]`
row removed (component is now defined in the dedicated file).
4. `specs/m/mozjs128/mozjs128.spec` and `specs/m/mozjs128/sources`
-- regenerated. The `sources` manifest now carries the modified-
tarball SHA-512 as the sole entry for that filename.
5. `locks/mozjs128.lock` -- refreshed input-fingerprint.
Render validation
-----------------
- `azldev comp update -p mozjs128` -> CHANGED=true; new
input-fingerprint reflects the comp.toml migration.
- `azldev comp render -p mozjs128` -> STATUS=ok; the audit WARN log
confirms the upstream `firefox-128.11.0esr.source.tar.xz` entry
was swapped from upstream SHA-512 `80af64c1...092d279` to the
modified-tarball SHA-512 `a79fe02e...9020f5bc`.
- `specs/m/mozjs128/sources` carries the modified-tarball SHA-512
as the sole entry for that filename.
- Mock build (`%prep`) succeeds.
- Mock build (`%build`) past the SpiderMonkey configure step:
earlier keep-list iterations dropped the top-level `configure.py`
Python entrypoint that `js/src/configure` execs into (added to
`KEEP_TOP`) and the `mozfile` helper module at
`testing/mozbase/mozfile` that the build's `find_program`
imports (restored via a `NESTED_KEEP=(testing/mozbase)`
re-extract step after the top-level strip).
PawelWMS
force-pushed
the
pawelwi/mozjs128-strip-source
branch
from
0650fc3 to
1175acc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: AB#19867
Koji build
Summary
The
mozjs128SRPM builds the SpiderMonkey JavaScript engine from the full upstream Firefox ESR source tarball (firefox-128.11.0esr.source.tar.xz, ~500 MB). The%buildblock only consumesjs/src/, but the SRPM payload contains every Firefox subtree — including malware-scanner-tripping fixtures the automated package-signing pipeline rejects (aes_archive.zip, NSIS installer DLL block, thelzma_sdk/google/test_data/encrypted{,_header}.7zfixtures, the bundledsetuptools/pipWindows PE launcher stubs, thetoolkit/components/telemetry/tests/unit/*.dllPE blobs, and more).This PR replaces Source0 with a deterministically-repacked Firefox source tarball that keeps only the subtrees
mozjs128's%buildand%installactually use, and drops everything else.Changes
base/comps/mozjs128/mozjs128.comp.toml(new) — dedicated component file with a single[[components.mozjs128.source-files]]block carrying:filename = "firefox-128.11.0esr.source.tar.xz"(matches upstream so the spec's Source0 line does not need editing).hashof the repacked tarball.origin.uripointing at the lookasiderepocontainer under thepkgs_modified/prefix.replace-upstream = true+replace-reason = "..."to swap the same-named upstream entry in the Fedorasourcesmanifest in place.base/comps/mozjs128/modify_source.sh(new) — deterministic strip-and-repack script. Downloads the upstream Firefox ESR tarball, verifies its SHA-512, deletes everything outside the keep-list, repacks withtar --sort=name --mtime=...+xz -T 1 -9 --block-size=...(single-threaded for determinism), prints the new SHA-512 plus a ready-to-pasteaz storage blob uploadcommand. Output lives atbase/build/work/scratch/mozjs128/.base/comps/components.toml— inline[components.mozjs128]row removed (component is now defined in the dedicated file).specs/m/mozjs128/sourcesandspecs/m/mozjs128/mozjs128.spec— regenerated.locks/mozjs128.lock— refreshed.Keep-list
LICENSE,Cargo.toml,Cargo.lock,moz.configure,build/,config/,js/,mfbt/,memory/,mozglue/,python/mozbuild/,third_party/. Plus an additionaljs/-internal strip ofjs/src/fuzz-tests/,js/src/devtools/automation/variants/,js/src/octane/,js/src/ctypes/libffi/(matches the existing%prep-timerm -rfdeletions, but removes them from the SRPM payload rather than at build time).third_party/is in the keep-list becausemozjs128.specdoeschmod -x third_party/rust/bumpalo/src/lib.rsin%prep(would fail underset -eif stripped), and the SpiderMonkey cargo build underjs/src/links against vendored Rust crates fromthird_party/rust/.Validation
4cec711d46502beea27d0e96e95c1de70a53139bd9c71dcc5f476815a1b3aa0bab3613f4883c33707938801660d74463b112817c4a68dc51993e2a0ad558d19f(deterministic).origin.urireferences.