Skip to content

feat: Source-prep origin summary#194

Draft
Tonisal-byte wants to merge 1 commit into
microsoft:mainfrom
Tonisal-byte:asalinas/source-origin-summary
Draft

feat: Source-prep origin summary#194
Tonisal-byte wants to merge 1 commit into
microsoft:mainfrom
Tonisal-byte:asalinas/source-origin-summary

Conversation

@Tonisal-byte
Copy link
Copy Markdown
Contributor

The prepare-sources command now returns a structured report listing every file that was downloaded during source preparation and where it came from. This enables downstream tools to audit and log the origin of all source files used in a build.

Each entry in the report includes the filename, origin type, download URL, and hash information. Two origin types are tracked:

  • lookaside-url: file downloaded from a lookaside cache
  • configured-origin-url: file downloaded from an explicitly configured origin URL in the component's source-files configuration

Files that already existed on disk (not downloaded this run) are omitted from the report. Git-tracked files (spec, patches), local sources, and SRPM-extracted files are out of scope -- only actual network downloads are reported.

The report is returned as structured output from the command, so it can be consumed via any supported output format:

 azldev component prep-sources -p <component> -o <dir> --force -O json

Copilot AI review requested due to automatic review settings May 15, 2026 23:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds provenance reporting for source preparation so component prepare-sources can return structured information about source files downloaded from lookaside caches or configured origin URLs.

Changes:

  • Adds SourceProvenance/ProvenanceReport types and propagates provenance through source manager, Fedora lookaside extraction, and source preparation.
  • Updates prepare-sources to return downloaded source provenance through command output.
  • Updates affected interfaces, mocks, and tests to account for the new return values.

Reviewed changes

Copilot reviewed 20 out of 22 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
internal/app/azldev/cmds/component/preparesources.go Returns provenance entries from the prepare-sources command.
internal/app/azldev/cmds/component/render.go Ignores the new provenance result for render flow.
internal/app/azldev/cmds/downloadsources/downloadsources.go Ignores provenance for standalone source downloading.
internal/app/azldev/cmds/downloadsources/downloadsources_test.go Updates downloader mock returns.
internal/app/azldev/core/componentbuilder/componentbuilder.go Ignores provenance during SRPM source prep.
internal/app/azldev/core/componentbuilder/componentbuilder_test.go Updates source manager mocks.
internal/app/azldev/core/sources/provenance.go Adds source preparation provenance report type.
internal/app/azldev/core/sources/sourceprep.go Aggregates provenance from file and component source fetches.
internal/app/azldev/core/sources/sourceprep_test.go Adds provenance aggregation tests and updates mocks.
internal/providers/sourceproviders/fedorasource/fedorasource.go Returns download records from Fedora sources extraction.
internal/providers/sourceproviders/fedorasource/fedorasource_test.go Updates extractor call sites.
internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go Regenerates mock return values.
internal/providers/sourceproviders/fedorasourceprovider.go Converts lookaside downloads into provenance.
internal/providers/sourceproviders/fedorasourceprovider_test.go Updates provider mock expectations.
internal/providers/sourceproviders/identityprovider_test.go Updates no-op downloader stub.
internal/providers/sourceproviders/provenance.go Adds provider-level provenance model and conversion helper.
internal/providers/sourceproviders/rpmcontentsprovider.go Adapts RPM provider interface to new return type.
internal/providers/sourceproviders/rpmcontentsprovider_test.go Updates RPM provider call sites.
internal/providers/sourceproviders/sourcemanager.go Propagates provenance through source file/component fetching.
internal/providers/sourceproviders/sourcemanager_test.go Updates source manager tests for new return values.
internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go Regenerates source manager mock return values.
internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks_noop.go Updates no-op mock defaults.
Files not reviewed (2)
  • internal/providers/sourceproviders/fedorasource/fedorasource_test/fedorasource_mocks.go: Language not supported
  • internal/providers/sourceproviders/sourceproviders_test/sourcemanager_mocks.go: Language not supported
Comments suppressed due to low confidence (4)

internal/app/azldev/cmds/component/preparesources.go:61

  • There is no command-level test covering the new structured return value from 'prepare-sources'. Existing command tests only check wiring/no-match behavior, so a regression that returns nil, the wrong shape, or drops hash fields would not be caught.
			report, err := PrepareComponentSources(env, &options)
			if err != nil {
				return nil, err
			}

			return report.Sources, nil

internal/providers/sourceproviders/fedorasource/fedorasource.go:315

  • In dry-run mode the HTTP downloader returns success without writing or fetching the file, but this branch still appends a SourceDownload entry. That makes the provenance report claim a file was actually downloaded even though '--dry-run' explicitly avoided the network/file write.
		downloads = append(downloads, SourceDownload{
			Filename: sourceFile.fileName,
			URL:      sourceFile.uri,
			HashType: sourceFile.hashType,
			Hash:     sourceFile.expectedHash,
		})

internal/providers/sourceproviders/sourcemanager.go:343

  • In dry-run mode downloadAndValidate can return nil without performing a network download or creating the file, but this success path still returns provenance. The report therefore records lookaside downloads that did not actually happen under '--dry-run'.
			return &SourceProvenance{
				Filename:   fileRef.Filename,
				OriginType: SourceOriginLookaside,
				URL:        sourceURL,
				HashType:   fileRef.HashType,
				Hash:       fileRef.Hash,
			}, nil

internal/providers/sourceproviders/sourcemanager.go:375

  • The same dry-run issue applies to configured-origin downloads: downloadFromOrigin can succeed without a real download when '--dry-run' is set, but the provenance report still records a configured-origin-url entry as if the file was fetched.
	return &SourceProvenance{
		Filename:   fileRef.Filename,
		OriginType: SourceOriginURL,
		URL:        originURL,
		HashType:   fileRef.HashType,
		Hash:       fileRef.Hash,
	}, nil

component.GetName(), err)
}

allProvenance = append(allProvenance, fileProv...)
Comment on lines +56 to +61
report, err := PrepareComponentSources(env, &options)
if err != nil {
return nil, err
}

return report.Sources, nil
Comment on lines +310 to +315
downloads = append(downloads, SourceDownload{
Filename: sourceFile.fileName,
URL: sourceFile.uri,
HashType: sourceFile.hashType,
Hash: sourceFile.expectedHash,
})
Comment on lines +337 to +343
return &SourceProvenance{
Filename: fileRef.Filename,
OriginType: SourceOriginLookaside,
URL: sourceURL,
HashType: fileRef.HashType,
Hash: fileRef.Hash,
}, nil
Comment on lines +36 to +39
HashType fileutils.HashType `json:"hashType,omitempty" table:"-"`

// Hash is the hex-encoded hash value used to validate the download.
Hash string `json:"hash,omitempty" table:"-"`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants