-
Notifications
You must be signed in to change notification settings - Fork 75
RFC: Remove Stacks & Mixins #172
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 6 commits
d16f1a9
1b55b44
5e7d9ff
df7d715
02cb22a
f888612
ce16a04
b876676
891104e
e2587fb
0dd5899
b1f606c
4299b4b
ef6d531
ff4c94a
3fe2b89
e24bcad
85545b5
ad051c4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| # Meta | ||
| [meta]: #meta | ||
| - Name: Remove Stacks & Mixins | ||
| - Start Date: 2021-06-30 | ||
| - Author(s): sclevine | ||
| - RFC Pull Request: (leave blank) | ||
| - CNB Pull Request: (leave blank) | ||
| - CNB Issue: (leave blank) | ||
| - Supersedes: [RFC0069](https://github.com/buildpacks/rfcs/blob/main/text/0069-stack-buildpacks.md), [RFC#167](https://github.com/buildpacks/rfcs/pull/167), many others | ||
|
|
||
| # Summary | ||
| [summary]: #summary | ||
|
|
||
| *NOTE: This proposal is part of a larger initiative to reduce complexity originally outlined in https://github.com/buildpacks/rfcs/pull/167* | ||
|
|
||
| This RFC proposes that we remove the "stack" and "mixin" concepts from the project and replace them with existing constructs in the container image ecosystem such as base images, Dockerfiles, and OS packages. | ||
|
|
||
| # Motivation | ||
| [motivation]: #motivation | ||
|
|
||
| The "stack" and "mixin" concepts add unnecessary complexity to the project and make it difficult for new users and contributors to understand how buildpacks work. Compatibility guarantees that are strongly enforced by the stack contract could be replaced with metadata validations and warnings. | ||
|
|
||
| # What it is | ||
| [what-it-is]: #what-it-is | ||
|
|
||
| Summary of changes: | ||
| - Remove mixins | ||
| - Replace stack metadata (including stack IDs) with canonical OS metadata. | ||
|
jromero marked this conversation as resolved.
|
||
|
|
||
| # How it Works | ||
| [how-it-works]: #how-it-works | ||
|
|
||
| ## Base Image Metadata | ||
|
sclevine marked this conversation as resolved.
|
||
|
|
||
| Instead of a stack ID, runtime and build-time base images are labeled with the following canonicalized metadata: | ||
|
jromero marked this conversation as resolved.
Outdated
jromero marked this conversation as resolved.
Outdated
|
||
| - OS (e.g., "linux", `$GOOS`) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are these strings able to be anything? Could I, if I wanted to, create a
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OS and Architecture would be restricted to |
||
| - Architecture (e.g., "x86_64", `$GOARCH`) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Off-topic: Is there any plan/idea for buildpacks to support building multi-platform images, such that a single image manifest can describe an image for
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No current RFC, but I definitely agree that we should eventually support exporting multi-platform manifest indexes. The BuildKit frontend might help us with this, via buildx: buildpacks/pack#768 For non-local builds, we could extend the current exporter to accept a set of layers for each OS/ARCH combination. In either case, we'd probably want to pass standardized env vars (e.g.,
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This is hard to do well given that (some exceptions wrt docker-for-mac notwithstanding) the build takes place in a docker container and a given daemon only supports a single os/arch. You could theoretically have a buildpacks execute on one target while producing launch layers that are valid for a different target but this puts a lot of responsibility on the buildpack author and comes with a lot of gotchas. e.g
imho this is unlikely to end well
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I'm not opposed to this. Cross-compilation seems like a reasonable feature request to me. We would probably want to pass "target" metadata (e.g., from a run image label) to the buildpacks. When multiple run images are provided, buildpacks could produce multiple versions of the same layer for different architectures. |
||
| - Distribution (optional) (e.g., "ubuntu", `$ID`) | ||
| - Version (optional) (e.g., "18.04", `$VERSION_ID`) | ||
|
|
||
| OS and Architecture must be valid identifiers as defined [here](https://golang.org/doc/install/source#environment). | ||
| For Linux-based images, each field should be canonicalized against values specified in `/etc/os-release` (`$ID` and `$VERSION_ID`). | ||
|
hone marked this conversation as resolved.
Outdated
sclevine marked this conversation as resolved.
Outdated
|
||
|
|
||
| The `stacks` list in `buildpack.toml` is replaced by a `targets` list, where each entry corresponds to a different buildpack image that is exported into a [manifest index](https://github.com/opencontainers/image-spec/blob/master/image-index.md). Each entry may contain multiple valid values for Distribution and/or Version, but only a single OS and Architecture. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would we support a wildcards in the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does the omission of
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. At least one OS/arch combination is necessary, but it doesn't need to include a distro/version. I'm hesitant to make OS/arch (e.g., linux x86) optional, because OCI images must contain these fields.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we were to support wildcards, could the builder or platform provide the OS/arch labels for the image? Surely it can infer this from the runtime base image being targeted.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I just meant for the buildpack image. If we supported a single buildpack image that could run on multiple architectures, it's
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My goal here is to eliminate boiler plate, and to make buildpack authorship simpler. Not necessarily to support multiple archs
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. With
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Actually, I think there are plenty of multi-OS or multi-architecture buildpacks. Maybe Maven, Gradle, etc. Most buildpacks that run something atop a language VM. Maybe https://github.com/jkutner/no-secrets-buildpack, which just scans a repos for secrets (doesn't even provide or install anything)
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added default values (matching OS & x86_64) to reduce boilerplate. Now buildpack authors only need to care about targets when they want to restrict usage of their buildpack to a specific distro/version, or when they need support for less common architectures. The OS is detected automatically ( OCI images must have a single OS and single architecture field, and I don't think we should create buildpack images for more than one architecture automatically. Can we iterate on this in the future? |
||
|
|
||
| App image builds fail if the build image and selected run image have mismatched metadata. We may introduce flags or additional labels to skip this validation (e.g., for cross-compilation or minimal runtime base images). An image without a specified Distribution is compatible with images specifying any Distribution. An image specifying a Distribution without a Version is compatible with images specifying any Versions of that Distribution. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Haven't thought this through all the way but would it make sense to this in the build plan as well. E.g. a buildpack could specify in
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Is this what would replace wildcard stack support, i.e.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I'm not sure this needs to be dynamic. I wonder if this could be part of the multi-platform interface? See: #172 (comment)
The requirement is just describing the contract between the build and run images. Wildcard stack support is achieved by leaving off Distribution and Version entirely. (There is no wildcard support for OS/Architecture.)
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. just updating this thread in case someone comes along in two more years, but anyway it turned out while implementing we did make an explicit extension for "*" OS/Architecture. |
||
|
|
||
| When an app image is rebased, `pack rebase` will fail if the new run image and previous run image have mismatched metadata. This check may be skipped for Distribution and Version by passing a new `--force` flag to `pack rebase`. | ||
|
sclevine marked this conversation as resolved.
Outdated
|
||
|
|
||
|
Comment on lines
+68
to
+69
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. /queue-issue buildpacks/lifecycle "Rebaser should validate new run image metadata against old run image metadata" epic/dockerfiles type/enhancement
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. /queue-issue buildpacks/pack "Add --force to rebase operation" |
||
| #### Example: buildpack.toml `targets` table | ||
|
|
||
| ```toml | ||
|
hone marked this conversation as resolved.
|
||
| [[targets]] | ||
| os = "linux" | ||
| arch = "x86_64" | ||
| [[targets.distros]] | ||
| name = "ubuntu" | ||
| versions = ["18.04", "20.04"] | ||
|
|
||
| [[targets]] | ||
| os = "linux" | ||
| arch = "x86_64" | ||
|
ekcasey marked this conversation as resolved.
|
||
| [[targets.distros]] | ||
| name = "ubuntu" | ||
| versions = ["14.04", "16.04"] | ||
| ``` | ||
|
|
||
| ## Mixins | ||
|
|
||
| Mixins are no longer used. If an SBoM is available, platforms may warn when, e.g., a rebase operation would change the available packages. | ||
|
|
||
| ### Example: CycloneDX SBoM | ||
|
|
||
| ```json | ||
| { | ||
| "bomFormat": "CycloneDX", | ||
| "specVersion": "1.3", | ||
| "version": 1, | ||
| "components": [ | ||
| { | ||
| "type": "library", | ||
| "name": "curl", | ||
| "version": "1.2.3", | ||
| "purl": "pkg:deb/ubuntu/curl@1.2.3" | ||
| }, | ||
| { | ||
| "type": "library", | ||
| "name": "libcurl", | ||
| "version": "4.5.6", | ||
| "purl": "pkg:deb/ubuntu/libcurl@4.5.6" | ||
| }, | ||
| ... | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| ### Validations | ||
|
|
||
| Buildpack base image metadata specified in `buildpack.toml`'s `targets` list are validated against the runtime and build-time base images. | ||
|
jromero marked this conversation as resolved.
|
||
|
|
||
| Runtime and build-time base image packages are no longer validated against each other. | ||
|
|
||
| If an SBoM is available, `pack rebase` will fail if packages are removed from the new runtime base image. | ||
|
sclevine marked this conversation as resolved.
Outdated
|
||
| This check may be skipped by passing a new `--force` flag to `pack rebase`. | ||
| However, this validation is not enforced by the specification. | ||
|
|
||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we need to address the migration path in this RFC
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Addressed, let me know if you think I covered the relevant migration scenarios
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for adding this! Would we also continue to allow
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Definitely, I can add that as well
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
@jkutner This makes me think |
||
| # Drawbacks | ||
| [drawbacks]: #drawbacks | ||
|
|
||
| - Involves breaking changes. | ||
|
|
||
| # Alternatives | ||
| [alternatives]: #alternatives | ||
|
|
||
| - Keep stacks. | ||
|
sclevine marked this conversation as resolved.
|
||
| - Make the above changes but keep some existing terminology: stacks, build-image, run-image. | ||
|
jromero marked this conversation as resolved.
|
||
| - Continue to allow buildpacks to specify package requirements (e.g., by PURL instead of mixins) | ||
|
|
||
| # Unresolved Questions | ||
|
sclevine marked this conversation as resolved.
|
||
| [unresolved-questions]: #unresolved-questions | ||
|
|
||
| - How will migration work? Can we make new base images compatible with older buildpacks? Can we make newer buildpacks compatible with older stacks? | ||
| - What should builder.toml (and similar stack-dependent config files) look like? What should assets look like? Note: these could be decided in subsequent subteam RFCs / spec PRs. | ||
|
sclevine marked this conversation as resolved.
jromero marked this conversation as resolved.
|
||
|
|
||
|
sclevine marked this conversation as resolved.
|
||
| # Spec. Changes (OPTIONAL) | ||
| [spec-changes]: #spec-changes | ||
|
|
||
| This RFC requires extensive changes to all specifications. | ||
Uh oh!
There was an error while loading. Please reload this page.