feat: Source-prep origin summary#194
Draft
Tonisal-byte wants to merge 1 commit into
Draft
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds provenance reporting for source preparation so component prepare-sources can return structured information about source files downloaded from lookaside caches or configured origin URLs.
Changes:
- Adds
SourceProvenance/ProvenanceReporttypes and propagates provenance through source manager, Fedora lookaside extraction, and source preparation. - Updates
prepare-sourcesto return downloaded source provenance through command output. - Updates affected interfaces, mocks, and tests to account for the new return values.
Reviewed changes
Copilot reviewed 20 out of 22 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
internal/app/azldev/cmds/component/preparesources.go |
Returns provenance entries from the prepare-sources command. |
internal/app/azldev/cmds/component/render.go |
Ignores the new provenance result for render flow. |
internal/app/azldev/cmds/downloadsources/downloadsources.go |
Ignores provenance for standalone source downloading. |
internal/app/azldev/cmds/downloadsources/downloadsources_test.go |
Updates downloader mock returns. |
internal/app/azldev/core/componentbuilder/componentbuilder.go |
Ignores provenance during SRPM source prep. |
internal/app/azldev/core/componentbuilder/componentbuilder_test.go |
Updates source manager mocks. |
internal/app/azldev/core/sources/provenance.go |
Adds source preparation provenance report type. |
internal/app/azldev/core/sources/sourceprep.go |
Aggregates provenance from file and component source fetches. |
internal/app/azldev/core/sources/sourceprep_test.go |
Adds provenance aggregation tests and updates mocks. |
internal/providers/sourceproviders/fedorasource/fedorasource.go |
Returns download records from Fedora sources extraction. |
internal/providers/sourceproviders/fedorasource/fedorasource_test.go |
Updates extractor call sites. |
internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go |
Regenerates mock return values. |
internal/providers/sourceproviders/fedorasourceprovider.go |
Converts lookaside downloads into provenance. |
internal/providers/sourceproviders/fedorasourceprovider_test.go |
Updates provider mock expectations. |
internal/providers/sourceproviders/identityprovider_test.go |
Updates no-op downloader stub. |
internal/providers/sourceproviders/provenance.go |
Adds provider-level provenance model and conversion helper. |
internal/providers/sourceproviders/rpmcontentsprovider.go |
Adapts RPM provider interface to new return type. |
internal/providers/sourceproviders/rpmcontentsprovider_test.go |
Updates RPM provider call sites. |
internal/providers/sourceproviders/sourcemanager.go |
Propagates provenance through source file/component fetching. |
internal/providers/sourceproviders/sourcemanager_test.go |
Updates source manager tests for new return values. |
internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go |
Regenerates source manager mock return values. |
internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks_noop.go |
Updates no-op mock defaults. |
Files not reviewed (2)
- internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
- internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go: Language not supported
Comments suppressed due to low confidence (4)
internal/app/azldev/cmds/component/preparesources.go:61
- There is no command-level test covering the new structured return value from 'prepare-sources'. Existing command tests only check wiring/no-match behavior, so a regression that returns nil, the wrong shape, or drops hash fields would not be caught.
report, err := PrepareComponentSources(env, &options)
if err != nil {
return nil, err
}
return report.Sources, nil
internal/providers/sourceproviders/fedorasource/fedorasource.go:315
- In dry-run mode the HTTP downloader returns success without writing or fetching the file, but this branch still appends a SourceDownload entry. That makes the provenance report claim a file was actually downloaded even though '--dry-run' explicitly avoided the network/file write.
downloads = append(downloads, SourceDownload{
Filename: sourceFile.fileName,
URL: sourceFile.uri,
HashType: sourceFile.hashType,
Hash: sourceFile.expectedHash,
})
internal/providers/sourceproviders/sourcemanager.go:343
- In dry-run mode downloadAndValidate can return nil without performing a network download or creating the file, but this success path still returns provenance. The report therefore records lookaside downloads that did not actually happen under '--dry-run'.
return &SourceProvenance{
Filename: fileRef.Filename,
OriginType: SourceOriginLookaside,
URL: sourceURL,
HashType: fileRef.HashType,
Hash: fileRef.Hash,
}, nil
internal/providers/sourceproviders/sourcemanager.go:375
- The same dry-run issue applies to configured-origin downloads: downloadFromOrigin can succeed without a real download when '--dry-run' is set, but the provenance report still records a configured-origin-url entry as if the file was fetched.
return &SourceProvenance{
Filename: fileRef.Filename,
OriginType: SourceOriginURL,
URL: originURL,
HashType: fileRef.HashType,
Hash: fileRef.Hash,
}, nil
| component.GetName(), err) | ||
| } | ||
|
|
||
| allProvenance = append(allProvenance, fileProv...) |
Comment on lines
+56
to
+61
| report, err := PrepareComponentSources(env, &options) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return report.Sources, nil |
Comment on lines
+310
to
+315
| downloads = append(downloads, SourceDownload{ | ||
| Filename: sourceFile.fileName, | ||
| URL: sourceFile.uri, | ||
| HashType: sourceFile.hashType, | ||
| Hash: sourceFile.expectedHash, | ||
| }) |
Comment on lines
+337
to
+343
| return &SourceProvenance{ | ||
| Filename: fileRef.Filename, | ||
| OriginType: SourceOriginLookaside, | ||
| URL: sourceURL, | ||
| HashType: fileRef.HashType, | ||
| Hash: fileRef.Hash, | ||
| }, nil |
Comment on lines
+36
to
+39
| HashType fileutils.HashType `json:"hashType,omitempty" table:"-"` | ||
|
|
||
| // Hash is the hex-encoded hash value used to validate the download. | ||
| Hash string `json:"hash,omitempty" table:"-"` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The prepare-sources command now returns a structured report listing every file that was downloaded during source preparation and where it came from. This enables downstream tools to audit and log the origin of all source files used in a build.
Each entry in the report includes the filename, origin type, download URL, and hash information. Two origin types are tracked:
Files that already existed on disk (not downloaded this run) are omitted from the report. Git-tracked files (spec, patches), local sources, and SRPM-extracted files are out of scope -- only actual network downloads are reported.
The report is returned as structured output from the command, so it can be consumed via any supported output format: