Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 49 additions & 55 deletions .github/CI-ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,8 @@ PR opened/updated
├──► pr-build-main.yml (Build and test)
│ │
│ ├── regen.sh (full build, no tests)
│ ├── incremental-build (test affected modules)
│ │ ├── File-path analysis
│ │ ├── POM dependency analysis
│ │ └── Extra modules (/component-test)
│ ├── regen.sh (build + test via Scalpel skip-tests)
│ ├── generate test report (post-build comment)
│ │
│ └──► pr-test-commenter.yml (post unified comment)
Expand All @@ -36,10 +33,11 @@ PR comment: /component-test kafka http
- **Trigger**: `pull_request` (main branch), `workflow_dispatch`
- **Matrix**: JDK 17, 21, 25 (25 is experimental)
- **Steps**:
1. Full build via `regen.sh` (`mvn install -DskipTests -Pregen`)
2. Check for uncommitted generated files
3. Run incremental tests (only affected modules)
4. Upload test comment as artifact
1. Check `skip-tests` label
2. Build and test via `regen.sh` (single Maven invocation with Scalpel)
3. Check for uncommitted generated files
4. Generate test report from Scalpel's JSON output
5. Upload test comment as artifact
- **Inputs** (workflow_dispatch): `pr_number`, `pr_ref`, `extra_modules`, `skip_full_build`

### `pr-test-commenter.yml` — Post CI test comment
Expand Down Expand Up @@ -70,8 +68,8 @@ PR comment: /component-test kafka http

- **Status**: Temporarily disabled (INFRA-27808 — SonarCloud quality gate adjustment pending)
- **Trigger**: `pull_request` (main branch) → `workflow_run` on SonarBuild completion
- **Why two workflows**: `sonar-build.yml` runs in PR context (builds with JaCoCo coverage on core modules, uploads compiled classes artifact), `sonar-scan.yml` runs via `workflow_run` with secrets access to run the Sonar scanner and post results
- **Coverage scope**: Currently limited to core modules (`camel-api`, `camel-core`, etc.) and `coverage` aggregator. Component coverage planned for future integration with `incremental-build.sh` module detection
- **Why two workflows**: `sonar-build.yml` runs in PR context (builds with JaCoCo coverage via Scalpel, uploads compiled classes artifact), `sonar-scan.yml` runs via `workflow_run` with secrets access to run the Sonar scanner and post results
- **Coverage scope**: Scalpel dynamically detects affected modules and runs tests with JaCoCo coverage on them — no hardcoded module list

### Other workflows

Expand All @@ -87,22 +85,19 @@ PR comment: /component-test kafka http

### `incremental-build`

The core test runner. Determines which modules to test using:
Post-build test report generator. Reads Scalpel's JSON report (`target/scalpel-report.json`) and generates a PR comment showing:

1. **File-path analysis**: Maps changed files to Maven modules
2. **POM dependency analysis** (dual detection):
- **Grep-based**: For `parent/pom.xml` changes, detects property changes and finds modules that explicitly reference the affected properties via `${property}` in their `pom.xml` files
- **Scalpel-based**: Uses [Maveniverse Scalpel](https://github.com/maveniverse/scalpel) (Maven extension) for effective POM model comparison — catches managed dependencies, plugin version changes, BOM imports, and transitive dependency impacts that the grep approach misses
3. **Extra modules**: Additional modules passed via `/component-test`

Both detection methods run in parallel. Their results are merged (union), deduplicated, and tested. If Scalpel fails (build error, runtime error), the script falls back to grep-only with no regression.
1. **Directly affected modules** with detection reasons (`SOURCE_CHANGE`, `POM_CHANGE`, `TRANSITIVE_DEPENDENCY`, `MANAGED_PLUGIN`, `TEST_CHANGE`)
2. **Changed properties, managed dependencies, managed plugins**
3. **Downstream modules** tested as dependents (collapsible)
4. **Upstream modules** compiled but with tests skipped (collapsible)
5. **Extra modules** from `/component-test`

The script also:

- Detects tests disabled in CI (`@DisabledIfSystemProperty(named = "ci.env.name")`)
- Applies an exclusion list for generated/meta modules
- Checks for excluded modules with associated integration tests (via `manual-it-mapping.txt`) and advises contributors to run them manually
- Generates a unified PR comment with all test information
- Parses test failures from surefire/failsafe reports

### `install-mvnd`

Expand All @@ -112,57 +107,56 @@ Installs the Maven Daemon (mvnd) for faster builds.

Installs system packages required for the build.

## PR Labels
## Scalpel: Automatic Test Selection

| Label | Effect |
| --- | --- |
| `skip-tests` | Skip all tests |
| `test-dependents` | Force testing dependent modules even if threshold exceeded |
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) is a Maven core extension that detects which modules are affected by a PR's changes and controls test execution accordingly.

## CI Environment

The CI sets `-Dci.env.name=github.com` via `MVND_OPTS` (in `install-mvnd`). Tests can use `@DisabledIfSystemProperty(named = "ci.env.name")` to skip flaky tests in CI. The test comment warns about these skipped tests.
### How it works

## POM Dependency Detection: Dual Approach
Scalpel is configured in `.mvn/extensions.xml` and runs automatically during Maven builds. In CI, it operates in **skip-tests** mode: all modules are compiled, but tests are only executed on affected modules.

### Grep-based detection (legacy)
Detection capabilities:
- **Source changes**: Files changed in a module's directory
- **POM changes**: Property, dependency, and plugin changes in any `pom.xml`
- **Transitive dependencies**: Changes that propagate through the dependency graph
- **Managed dependencies**: Version changes via `<dependencyManagement>` (even without explicit `<version>` in child modules)
- **Managed plugins**: Plugin version changes via `<pluginManagement>`
- **Property indirection**: `${property}` references resolved transitively

The grep approach searches for `${property-name}` references in module `pom.xml` files. It has known limitations:
### Configuration

1. **Managed dependencies without explicit `<version>`** — Modules inheriting versions via `<dependencyManagement>` without declaring `<version>${property}</version>` are missed.
2. **Maven plugin version changes** — Plugin version properties consumed in `parent/pom.xml` via `<pluginManagement>` are invisible to child modules.
3. **BOM imports** — Modules using artifacts from a BOM are not linked to the BOM version property.
4. **Transitive dependency changes** — Only direct property references are detected.
5. **Non-property version changes** — Structural `<dependencyManagement>` edits without property substitution are not caught.
Scalpel flags are set in `etc/scripts/regen.sh` when running in CI (`GITHUB_ACTIONS=true`):

### Scalpel-based detection (new)
| Flag | Value | Purpose |
| --- | --- | --- |
| `scalpel.mode` | `skip-tests` | Build all, test only affected |
| `scalpel.skipTestsForUpstream` | `true` | Don't test upstream-only modules |
| `scalpel.fetchBaseBranch` | `false` | Base branch pre-fetched by workflow (git CLI handles shallow clones better than JGit) |
| `scalpel.fullBuildTriggers` | *(empty)* | Override `.mvn/**` default |
| `scalpel.reportFile` | `target/scalpel-report.json` | JSON report for PR comment |
| `scalpel.impactedLog` | `target/scalpel-impacted.txt` | Simple module path list |

[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) is a Maven core extension that compares effective POM models between the base branch and the PR. It resolves all 5 grep limitations by:
### Developer machines

- Reading old POM files from the merge-base commit (via JGit)
- Comparing properties, managed dependencies, and managed plugins between old and new POMs
- Resolving the full transitive dependency graph to find all affected modules
- Detecting plugin version changes via `project.getBuildPlugins()` comparison
On developer machines, Scalpel is a no-op: without `GITHUB_BASE_REF` (set by GitHub Actions for PRs), no base branch is detected and Scalpel returns immediately. Zero impact on local builds.

Scalpel runs in **report mode** (`-Dscalpel.mode=report`), writing a JSON report to `target/scalpel-report.json` without modifying the Maven reactor. The report includes affected modules with reasons (`SOURCE_CHANGE`, `POM_CHANGE`, `TRANSITIVE_DEPENDENCY`, `MANAGED_PLUGIN`).
### Fail-safe

### Dual-detection strategy
If Scalpel encounters an error, `failSafe=true` (the default) causes it to fall back to a full build — all tests run. This means Scalpel errors never cause false negatives (missed tests).

Both methods run in parallel. Results are merged (union) before testing. This lets us:
## PR Labels

1. **Validate Scalpel** — Compare what each method detects across many PRs
2. **No regression** — If Scalpel fails, grep results are still used
3. **Gradual migration** — Once Scalpel is validated, grep can be removed
| Label | Effect |
| --- | --- |
| `incremental-skip-tests` | Skip all tests (pass `--skip-tests` to regen.sh) |

Scalpel is configured permanently in `.mvn/extensions.xml` (version `0.1.0`). On developer machines it is a no-op — without CI environment variables (`GITHUB_BASE_REF`), no base branch is detected and Scalpel returns immediately. The `mvn validate` with report mode adds ~60-90 seconds in CI.
## CI Environment

Note: the script overrides `fullBuildTriggers` to empty (`-Dscalpel.fullBuildTriggers=`) because Scalpel's default (`.mvn/**`) would trigger a full build whenever `.mvn/extensions.xml` itself changes (e.g., Dependabot bumping Scalpel).
The CI sets `-Dci.env.name=github.com` via `MVND_OPTS` (in `install-mvnd`). Tests can use `@DisabledIfSystemProperty(named = "ci.env.name")` to skip flaky tests in CI. The test comment warns about these skipped tests.

## Manual Integration Test Advisories

Some modules are excluded from CI's `-amd` expansion (the `EXCLUSION_LIST`) because they are generated code, meta-modules, or expensive integration test suites. When a contributor changes one of these modules, CI cannot automatically test all downstream effects.

The file `manual-it-mapping.txt` (co-located with the incremental build script) maps source modules to their associated integration test suites. When a changed module has a mapping entry, CI posts an advisory in the PR comment:
Some integration test suites are excluded from CI. When a contributor changes a module with a mapping entry in `manual-it-mapping.txt`, CI posts an advisory in the PR comment:

> You modified `dsl/camel-jbang/camel-jbang-core`. The related integration tests in `dsl/camel-jbang/camel-jbang-it` are excluded from CI. Consider running them manually:
>
Expand All @@ -184,4 +178,4 @@ All non-experimental JDK matrix entries (17, 21) upload the CI comment artifact

PR comments use HTML markers for upsert (create-or-update) behavior:

- `<!-- ci-tested-modules -->` — Unified test summary comment
- `<!-- ci-tested-modules -->` — Unified test summary comment
35 changes: 9 additions & 26 deletions .github/actions/incremental-build/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,40 +15,23 @@
# limitations under the License.
#

name: "Incremental Test Runner"
description: "Test only affected projects, using file-path analysis and POM dependency detection"
name: "Test Report Generator"
description: "Generate PR comment from Scalpel's change detection report"
inputs:
pr-id:
description: 'Id of the pull request (optional for push builds)'
extra-modules:
description: 'Additional modules tested (comma-separated paths, e.g. from /component-test)'
required: false
default: ''
github-token:
description: 'The github token to access to the API'
required: false
skip-mvnd-install:
description: 'Flag indicating whether the installation of mvnd should be skipped'
build-failed:
description: 'Whether the build step failed (true/false)'
required: false
default: 'false'
github-repo:
description: 'The GitHub repository name (example, apache/camel)'
required: false
default: ${{ github.repository }}
extra-modules:
description: 'Additional modules to test (comma-separated paths, e.g. from /component-test)'
required: false
default: ''
runs:
using: "composite"
steps:
- id: install-mvnd
uses: apache/camel/.github/actions/install-mvnd@main
with:
dry-run: ${{ inputs.skip-mvnd-install }}
- name: maven test
- name: Generate test report
env:
GITHUB_TOKEN: ${{ inputs.github-token }}
PR_ID: ${{ inputs.pr-id }}
GITHUB_REPO: ${{ inputs.github-repo }}
EXTRA_MODULES: ${{ inputs.extra-modules }}
BUILD_FAILED: ${{ inputs.build-failed }}
shell: bash
run: ${{ github.action_path }}/incremental-build.sh ${{ steps.install-mvnd.outputs.mvnd-dir }}/mvnd "$PR_ID" "$GITHUB_REPO" "$EXTRA_MODULES"
run: ${{ github.action_path }}/incremental-build.sh "$EXTRA_MODULES"
Loading
Loading