Skip to content

Gradle: local-projects in the serialized application model is nondeterministic across Gradle JVMs, busting the build cache #55619

Description

@TSFenwick

The Gradle application model serializes its reloadable workspace modules from a Set.copyOf(...), whose JDK iteration order is randomized per JVM. The serialized local-projects array can therefore come out in a different order from one JVM to the next, so the model file's bytes change. Every task that reads the model can compute a different build-cache key across builds.

DefaultApplicationModel stores the modules as Set.copyOf(builder.reloadableWorkspaceModules), and ApplicationModel.asMap() writes that set straight into the local-projects key. java.util.ImmutableCollections salts iteration order per JVM, so any model with two or more local projects can serialize in a different order from one JVM to the next. Every multi-module test model qualifies, since it lists the module under test plus each local project dependency.

The model file (build/quarkus/application-model/*.dat) is produced by QuarkusApplicationModelTask and is a content-hashed build-cache input of every Test task (registered as a file input) and of the cacheable quarkusGenerateCode, quarkusGenerateCodeTests, quarkusAppPartsBuild, and quarkusBuild tasks (each takes it as an @InputFile applicationModel). A long-lived Gradle daemon hides the problem because it reuses one JVM. Every CI build starts a fresh JVM, so those tasks miss the shared build cache on most runs even when every real input is identical.

Reproduction

Minimal standalone repro: https://github.com/TSFenwick/quarkus-gradle-repro-repo/tree/local-projects-order

:app applies the Quarkus plugin with :lib-core in main scope and :lib-testing in test scope, so its test model lists three local projects, enough for the order to vary.

./repro.sh 8        # model bytes: expect "NONDETERMINISTIC: N orderings" across fresh JVMs
./cache-key.sh 6    # build cache key: expect the key for :app:quarkusGenerateCodeTests to flip

cache-key.sh prints the build-cache key Gradle computes for quarkusGenerateCodeTests (which takes the test model as its applicationModel input) on each fresh JVM. Without the workaround the key tracks the array order, so the same task gets several keys, and any build whose key differs from the stored one misses the cache:

run 2: cacheKey=a6718fd0... "local-projects":["org.example:app::jar","org.example:lib-core::jar","org.example:lib-testing::jar"]
run 3: cacheKey=3a7b355c... "local-projects":["org.example:app::jar","org.example:lib-testing::jar","org.example:lib-core::jar"]
CACHE KEY UNSTABLE: 4 distinct keys for :app:quarkusGenerateCodeTests across 5 fresh JVMs

Both scripts accept -PsortModel, which applies a stopgap sort and makes the bytes and the key stable.

Fix

Serialize the set-valued fields in a stable order in ApplicationModel.asMap, either sorting before writing or holding them in an ordered collection. The model and every dependent cache key then stay stable while content is unchanged, and still change when content does. This is the same stability property #54822 argues for on a different field of the same model file.

A downstream sort in the Gradle plugin is possible but poor. Doing it in a doLast rewrites the model task's own declared output, which breaks Gradle parallel execution (org.gradle.parallel). Fixing it in the serializer avoids that entirely.

Environment

Quarkus 3.36.3, Gradle 9.6.1, JDK 25, on Linux and macOS.

Relates to

Found during the same Gradle build-cache investigation as #54822.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions