Skip to content

machine files: add [compilers] section for hermetic toolchain configuration#15783

Open
Prince781 wants to merge 5 commits into
mesonbuild:masterfrom
Prince781:dev/pferro/compilers-section
Open

machine files: add [compilers] section for hermetic toolchain configuration#15783
Prince781 wants to merge 5 commits into
mesonbuild:masterfrom
Prince781:dev/pferro/compilers-section

Conversation

@Prince781
Copy link
Copy Markdown
Contributor

@Prince781 Prince781 commented May 7, 2026

Adds a new [compilers] section to native and cross machine files that allows declaring compiler configuration explicitly, rather than relying on Meson to auto-detect it from a binary. This is primarily useful for hermetic toolchains where the compiler and its subprograms require a custom execution environment not present on the build host.

When a language is declared in this section, the user can override compiler properties that Meson would infer by running <compiler> --version. <lang>.type allows controlling the compiler class. <lang>.version is an optional override.

The remaining keys -- sysroot, no-default-includes, system-include-dirs, tool-search-paths, and subprocess-interpreter -- translate to compiler flags at setup time so that the binary entries stay clean:

  • sysroot -> --sysroot=<path> (GCC/Clang); ignored for MSVC-family
  • no-default-includes -> -nostdinc / /X
  • system-include-dirs -> -isystem <dir> / /imsvc <dir>
  • tool-search-paths -> -B <dir> (GCC/Clang)
  • subprocess-interpreter -> GCC only: Meson generates thin cc1/cc1plus/lto1 wrapper scripts in the build scratch directory that invoke the real subprograms via the specified ELF interpreter, making the build fully cacheable by ccache without requiring -wrapper or LD_LIBRARY_PATH.

Example for a self-contained GCC 12.2.0 bundled under sdk/:

[constants]
_sdk     = '@GLOBAL_SOURCE_ROOT@' / 'sdk'
_gcc     = _sdk / 'gcc-12.2.0'
_runtime = _sdk / 'runtime'    # ELF loader and companion shared libraries

[binaries]
c   = _gcc / 'bin' / 'x86_64-linux-gnu-gcc'
cpp = _gcc / 'bin' / 'x86_64-linux-gnu-g++'

[compilers]
c.type    = 'gcc'
c.version = '12.2.0'
c.subprocess-interpreter = [_runtime / 'lib64' / 'ld-linux-x86-64.so.2',
                              '--library-path', _runtime / 'lib64']

cpp.type    = 'gcc'
cpp.version = '12.2.0'
cpp.subprocess-interpreter = c.subprocess-interpreter

Fixes #12612

Prince781 and others added 3 commits May 6, 2026 21:14
Adds a new [compilers] section to native/cross machine files that allows
declaring compiler type, binary, version, and toolchain configuration
explicitly — without the user needing to know GCC-specific invocation
flags or write wrapper scripts.

Key features:
- <lang>.type    skip --version family detection, use declared type
- <lang>.binary  override [binaries] compiler path
- <lang>.version skip --version parsing, use declared version
- <lang>.ccache  control ccache wrapping (default: true = auto-detect)
- <lang>.subprocess-interpreter  GCC: generate cc1/cc1plus/lto1 wrapper
  scripts in meson-private/compiler-wrappers/ so hermetic GCC subprograms
  can find their shared libraries without LD_LIBRARY_PATH or -wrapper
- <lang>.sysroot, .no-default-includes, .system-include-dirs,
  .tool-search-paths  parsed and stored; flag emission in follow-up

The fast path in detect.py handles type='gcc' fully: generates subprocess
wrappers before _get_gnu_compiler_defines() runs (which requires cc1),
so hermetic builds succeed end-to-end.

17 unit tests cover: parsing, type/version/binary/ccache/flags, unknown
key warnings, error paths (bad type, bad ccache type, missing dot).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When [binaries] c already includes -B<libexec>, the GCC driver finds the
real cc1 before reaching the generated wrapper directory that was appended
last. Insert -B<wrapper_dir> right after the binary (index 0) so it is
searched before any -B flags inherited from the compiler command.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Emit --sysroot, -nostdinc, -B, and -isystem flags directly from the
[compilers] descriptor in the GCC fast path, so [binaries] c/cpp can be
a bare binary path without any flags embedded.

Also fix ccache auto-detection: when [binaries] has no ccache prefix but
[compilers] ccache=true (the default), detect ccache automatically.

Flag order: structured flags are appended after the binary, then
_generate_subprocess_wrappers is called (so it has -B<libexec> available
for --print-prog-name), then -B<wrapper_dir> is prepended before them.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Prince781 Prince781 requested review from dcbaker and jpakkane as code owners May 7, 2026 07:15
@Prince781 Prince781 changed the title machine files: add [compilers] section for hermetic toolchain configuration machine files: add [compilers] section for hermetic toolchain configuration May 7, 2026
Prince781 added 2 commits May 7, 2026 00:21
Drop unused CompilerTable import, remove duplicate `c, cpp` and
`linkers as lnk_mod` reimports inside the GCC fast path (the top-level
imports already cover them), and fix an over-indented continuation line.
Two ways to specify the compiler binary (`[binaries].<lang>` and
`[compilers].<lang>.binary`) is confusing and forces precedence rules.
Drop `binary` entirely; the binary path lives only in `[binaries]`.

The descriptor is still consulted in `_get_compilers()` for the ccache
override, which fixes mesonbuild#12612 (a bare `[binaries] cpp = 'clang++'` losing
ccache): with `[compilers] cpp.ccache = true` (the default), ccache is
auto-detected on PATH.

Also adjust the docs version tag to 1.12.0 and drop the fortran example.
@gurchetansingh
Copy link
Copy Markdown

gurchetansingh commented May 9, 2026

You might want to take a look at #15462 and #14147 for other takes on a similar problem.

From a brief skim, wouldn't it be nice to be able to download the SDK and compilers via a wrap file? That's what my proposal does:


[[wrap]]
name = "android_ndk"
source_url = "https://dl.google.com/android/repository/android-ndk-r29-linux.zip"
source_filename = "android-ndk-r29-linux.zip"
source_hash = "4abbbcdc842f3d4879206e9695d52709603e52dd68d3c1fff04b3b5e7a308ecf"

[[toolchain]]
name = "android_ndk_toolchain_aarch64"
wrap_name = "android_ndk"
ar = "toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar"
cc = "toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android35-clang"
cpp = "toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android35-clang++"
strip = "toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip"

[[toolchain]]
name = "android_ndk_toolchain_x86"
wrap_name = "android_ndk"
ar = "toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar"
cc = "toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android35-clang"
cpp = "toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android35-clang++"
strip = "toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip"

[[toolchain]]
name = "android_ndk_toolchain_x86_64"
wrap_name = "android_ndk"
ar = "toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar"
cc = "toolchains/llvm/prebuilt/linux-x86_64/bin/x86_64-linux-android35-clang"
cpp = "toolchains/llvm/prebuilt/linux-x86_64/bin/x86_64-linux-android35-clang++"
strip = "toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip"

[[platform]]
name = "android_arm64"
toolchain = "android_ndk_toolchain_aarch64"

[platform.sysroot]
wrap_name = "android_ndk"
path = "toolchains/llvm/prebuilt/linux-x86_64/sysroot"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

compiler override disables ccache

2 participants