feat: Source-prep origin summary#194
Open
Tonisal-byte wants to merge 5 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds provenance reporting for source preparation so component prepare-sources can return structured information about source files downloaded from lookaside caches or configured origin URLs.
Changes:
- Adds
SourceProvenance/ProvenanceReporttypes and propagates provenance through source manager, Fedora lookaside extraction, and source preparation. - Updates
prepare-sourcesto return downloaded source provenance through command output. - Updates affected interfaces, mocks, and tests to account for the new return values.
Reviewed changes
Copilot reviewed 20 out of 22 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
internal/app/azldev/cmds/component/preparesources.go |
Returns provenance entries from the prepare-sources command. |
internal/app/azldev/cmds/component/render.go |
Ignores the new provenance result for render flow. |
internal/app/azldev/cmds/downloadsources/downloadsources.go |
Ignores provenance for standalone source downloading. |
internal/app/azldev/cmds/downloadsources/downloadsources_test.go |
Updates downloader mock returns. |
internal/app/azldev/core/componentbuilder/componentbuilder.go |
Ignores provenance during SRPM source prep. |
internal/app/azldev/core/componentbuilder/componentbuilder_test.go |
Updates source manager mocks. |
internal/app/azldev/core/sources/provenance.go |
Adds source preparation provenance report type. |
internal/app/azldev/core/sources/sourceprep.go |
Aggregates provenance from file and component source fetches. |
internal/app/azldev/core/sources/sourceprep_test.go |
Adds provenance aggregation tests and updates mocks. |
internal/providers/sourceproviders/fedorasource/fedorasource.go |
Returns download records from Fedora sources extraction. |
internal/providers/sourceproviders/fedorasource/fedorasource_test.go |
Updates extractor call sites. |
internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go |
Regenerates mock return values. |
internal/providers/sourceproviders/fedorasourceprovider.go |
Converts lookaside downloads into provenance. |
internal/providers/sourceproviders/fedorasourceprovider_test.go |
Updates provider mock expectations. |
internal/providers/sourceproviders/identityprovider_test.go |
Updates no-op downloader stub. |
internal/providers/sourceproviders/provenance.go |
Adds provider-level provenance model and conversion helper. |
internal/providers/sourceproviders/rpmcontentsprovider.go |
Adapts RPM provider interface to new return type. |
internal/providers/sourceproviders/rpmcontentsprovider_test.go |
Updates RPM provider call sites. |
internal/providers/sourceproviders/sourcemanager.go |
Propagates provenance through source file/component fetching. |
internal/providers/sourceproviders/sourcemanager_test.go |
Updates source manager tests for new return values. |
internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go |
Regenerates source manager mock return values. |
internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks_noop.go |
Updates no-op mock defaults. |
Files not reviewed (2)
- internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
- internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go: Language not supported
Comments suppressed due to low confidence (4)
internal/app/azldev/cmds/component/preparesources.go:61
- There is no command-level test covering the new structured return value from 'prepare-sources'. Existing command tests only check wiring/no-match behavior, so a regression that returns nil, the wrong shape, or drops hash fields would not be caught.
report, err := PrepareComponentSources(env, &options)
if err != nil {
return nil, err
}
return report.Sources, nil
internal/providers/sourceproviders/fedorasource/fedorasource.go:315
- In dry-run mode the HTTP downloader returns success without writing or fetching the file, but this branch still appends a SourceDownload entry. That makes the provenance report claim a file was actually downloaded even though '--dry-run' explicitly avoided the network/file write.
downloads = append(downloads, SourceDownload{
Filename: sourceFile.fileName,
URL: sourceFile.uri,
HashType: sourceFile.hashType,
Hash: sourceFile.expectedHash,
})
internal/providers/sourceproviders/sourcemanager.go:343
- In dry-run mode downloadAndValidate can return nil without performing a network download or creating the file, but this success path still returns provenance. The report therefore records lookaside downloads that did not actually happen under '--dry-run'.
return &SourceProvenance{
Filename: fileRef.Filename,
OriginType: SourceOriginLookaside,
URL: sourceURL,
HashType: fileRef.HashType,
Hash: fileRef.Hash,
}, nil
internal/providers/sourceproviders/sourcemanager.go:375
- The same dry-run issue applies to configured-origin downloads: downloadFromOrigin can succeed without a real download when '--dry-run' is set, but the provenance report still records a configured-origin-url entry as if the file was fetched.
return &SourceProvenance{
Filename: fileRef.Filename,
OriginType: SourceOriginURL,
URL: originURL,
HashType: fileRef.HashType,
Hash: fileRef.Hash,
}, nil
Tonisal-byte
force-pushed
the
asalinas/source-origin-summary
branch
from
e96e346 to
db5ef8a
Compare
Tonisal-byte
force-pushed
the
asalinas/source-origin-summary
branch
from
db5ef8a to
f0e33d4
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 32 out of 33 changed files in this pull request and generated 13 comments.
Files not reviewed (1)
- internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
Tonisal-byte
force-pushed
the
asalinas/source-origin-summary
branch
2 times, most recently
from
c7aa2f7 to
e6aa63b
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 23 out of 25 changed files in this pull request and generated 3 comments.
Files not reviewed (2)
- internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
- internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go: Language not supported
Tonisal-byte
force-pushed
the
asalinas/source-origin-summary
branch
from
e6aa63b to
3304a2a
Compare
Tonisal-byte
force-pushed
the
asalinas/source-origin-summary
branch
from
3304a2a to
0cbdbcf
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 23 changed files in this pull request and generated 1 comment.
Files not reviewed (2)
- internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
- internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go: Language not supported
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 24 changed files in this pull request and generated 3 comments.
Files not reviewed (2)
- internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
- internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go: Language not supported
Tonisal-byte
force-pushed
the
asalinas/source-origin-summary
branch
from
c4a7803 to
704f3dc
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 24 changed files in this pull request and generated 1 comment.
Files not reviewed (2)
- internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
- internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go: Language not supported
Comments suppressed due to low confidence (1)
internal/providers/sourceproviders/sourcemanager.go:369
- This dry-run return happens before
downloadFromOriginvalidates the origin details, so entries withorigin.type = "uri"and an empty URI—or any unsupported origin type—now succeed in dry-run mode and only fail in a real run. Keep the validation path running in dry-run and only skip the actual download/provenance emission.
if m.dryRunnable.DryRun() {
return nil, nil //nolint:nilnil // dry-run — origin validated but no download.
704f3dc to
e46fe1a
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 24 changed files in this pull request and generated 4 comments.
Files not reviewed (2)
- internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
- internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go: Language not supported
Tonisal-byte
force-pushed
the
asalinas/source-origin-summary
branch
from
e46fe1a to
13382f5
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 24 changed files in this pull request and generated 3 comments.
Files not reviewed (2)
- internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
- internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go: Language not supported
Tonisal-byte
force-pushed
the
asalinas/source-origin-summary
branch
from
13382f5 to
b16bd11
Compare
reubeno
reviewed
May 21, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add SkipExistingFiles filter to copySourceDirectory so that FetchLocalComponent does not overwrite source files already downloaded by FetchFiles, matching the upstream provider behavior. Also suppress provenance reporting during dry-run and restrict hash enrichment to the overlay path to avoid incorrect audit data. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move dry-run guards after origin/disable-origins validation so configuration errors are still caught during dry runs. Add FetchFiles tests for lookaside and origin-fallback provenance to cover the returned metadata fields. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Extend RPMProvider.GetRPM to return download URL and populate provenance in RPMContentsProviderImpl - Remove unnecessary length check in ConvertDownloadsToProvenance - Add explanatory comment for SkipExistingFiles in local provider - Name the provenance return value in FedoraSourcesProvider.GetComponent - Run hash enrichment unconditionally (not gated by applyOverlays) - Update enrichment to always sync hashes from the finalized sources file so provenance reflects the post-overlay state - Add example JSON output to prepare-sources command description - Update tests and regenerate mocks Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Tonisal-byte
force-pushed
the
asalinas/source-origin-summary
branch
from
b16bd11 to
0715a13
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 23 out of 26 changed files in this pull request and generated 4 comments.
Files not reviewed (3)
- internal/providers/rpmprovider/rpmprovider_test/rpmprovider_mocks.go: Language not supported
- internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
- internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go: Language not supported
- Fix version suffix regex in doc generator to strip '0.0.0-devel' (matches versions with or without 'v' prefix) - Regenerate CLI docs without version churn - Derive RPM provenance filename from download URL instead of synthesizing from component name - Fix stale comment on enrichProvenanceWithResolvedHashes to reflect that hashes are always synced from the sources file Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Tonisal-byte
force-pushed
the
asalinas/source-origin-summary
branch
from
7e3b3c8 to
de093b9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The prepare-sources command now returns a structured report listing every file that was downloaded during source preparation and where it came from. This enables downstream tools to audit and log the origin of all source files used in a build.
Each entry in the report includes the filename, origin type, download URL, and hash information. Two origin types are tracked:
Files that already existed on disk (not downloaded this run) are omitted from the report. Git-tracked files (spec, patches), local sources, and SRPM-extracted files are out of scope -- only actual network downloads are reported.
The report is returned as structured output from the command, so it can be consumed via any supported output format: