Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
cb59802
chore: remove deadcode (stateful resources validation)
sai-ray Apr 29, 2026
529c184
chore: remove stateful resource validation tests
sai-ray Apr 29, 2026
92461bd
chore: remove unimplemented validations
sai-ray Apr 29, 2026
5a462b6
chore: scaffold retain.ts
sai-ray Apr 29, 2026
0c3d679
chore: wire retain in gen2-migration.ts
sai-ray Apr 29, 2026
247b6f8
chore: add stack walker
sai-ray Apr 30, 2026
e71d11a
chore: add buildRetainOperation()
sai-ray Apr 30, 2026
937113c
chore: call the added fucntions
sai-ray Apr 30, 2026
fc037df
chore: add validation helper
sai-ray Apr 30, 2026
632992a
chore: wire validator
sai-ray Apr 30, 2026
e83f086
chore: update implications
sai-ray Apr 30, 2026
afcb080
chore: rollback
sai-ray Apr 30, 2026
5f59ac2
chore: add tests
sai-ray Apr 30, 2026
2ffe582
chore: update tests
sai-ray Apr 30, 2026
939112c
Merge remote-tracking branch 'origin/dev' into sai/retain-command-for…
sai-ray May 1, 2026
66cc2c4
chore: minor UX improvements
sai-ray May 1, 2026
46481ad
chore: minor UX improvements
sai-ray May 1, 2026
5a0ad7b
chore: minor UX improvements
sai-ray May 1, 2026
9f2c97c
chore: minor UX improvements
sai-ray May 1, 2026
75d4c85
chore: minor UC improvements
sai-ray May 1, 2026
43b9d34
chore: minor UX improvements
sai-ray May 1, 2026
8b811ed
chore: minor UX imporovements
sai-ray May 1, 2026
3c1ce65
chore: minor UX improvements
sai-ray May 1, 2026
af19b24
chore: minor UX improvements
sai-ray May 1, 2026
de6fa6f
chore: minor UX improvements
sai-ray May 4, 2026
6584941
chore: fix retain validator rejecting nested stack re-evals
sai-ray May 4, 2026
dd26790
chore: update tests
sai-ray May 4, 2026
65038db
chore: major UX changes
sai-ray May 4, 2026
5057170
chore: walk stack hierarachy root-first
sai-ray May 4, 2026
10e7673
Merge remote-tracking branch 'origin/dev' into sai/retain-command-for…
sai-ray May 4, 2026
80c224d
chore: wire retain into gen2-migration e2e
sai-ray May 4, 2026
97f5dd2
chore: remove flaky formatting test
sai-ray May 4, 2026
83bd384
chore: remove marker, skip nested-stack refs, add unlock step
sai-ray May 5, 2026
ea149cc
chore: lazy changeset creation to avoid OBSOLETE transitions
sai-ray May 5, 2026
e22c857
chore: document retain subcommand
sai-ray May 5, 2026
5030944
chore: skip createChangeSet when retain already applied
sai-ray May 6, 2026
3525bcb
chore: port walker and broadened validator into lock
sai-ray May 6, 2026
1a58ad4
chore: add resource-classification map in lock
sai-ray May 6, 2026
16e67b5
chore: add retain-everything ops to lock.forward
sai-ray May 6, 2026
1cfdeb0
chore: implement buildRetainOperation in lock
sai-ray May 6, 2026
918d0e6
chore: inline classifier and label-push helpers in lock
sai-ray May 6, 2026
a97c725
chore: retire old per-resource retain loop from lock
sai-ray May 6, 2026
444bcee
chore: update lock tests for retain-everything
sai-ray May 6, 2026
cfd646f
chore: remove retain subcommand
sai-ray May 6, 2026
b982392
chore: add 3-layer hierarchy test for lock retain walk
sai-ray May 6, 2026
af1ab14
chore: broaden lock rollback drift whitelist for retain-everything
sai-ray May 7, 2026
1730b67
chore: add retain2 step for gen2-migration
sai-ray May 7, 2026
bf3541c
chore: rename retain2 to retain
sai-ray May 7, 2026
be6634e
docs: document retain step for gen2-migration
sai-ray May 7, 2026
470bdeb
docs: tighten retain doc and classifyStacks jsdoc
sai-ray May 7, 2026
ed668d4
chore: revert lock.ts to origin/dev
sai-ray May 7, 2026
e7f9434
docs: resolve readme conflict by taking dev credential refresh section
sai-ray May 7, 2026
8dae5e4
chore: address pr feedback on retain
sai-ray May 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .kiro/skills/gen2-migration/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ The E2E runs the following phases in order:
17. Sandbox redeploy
18. Gen1 tests + Gen2 tests (final)
19. Shared data tests
20. Retain
21. Gen1 tests + Gen2 tests (post-retain)

#### App directory

Expand Down
30 changes: 15 additions & 15 deletions docs/packages/amplify-cli/src/commands/gen2-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ the migration of Gen1 applications to Gen2. It exposes a step-based CLI workflow
through the complete migration process:

1. Assessing migration readiness,
2. Locking the Gen1 environment,
2. Locking the Gen1 environment (retains every resource in every Gen1 stack as part of this step),
3. Generating Gen2 code,
4. Refactoring CloudFormation stacks to move stateful resources,
5. Decommissioning the Gen1 environment.
5. Retaining every resource below root so the user can safely delete the Gen1 root stack.

The `assess` subcommand is handled separately from the step lifecycle — it is read-only and does not follow the `validate → execute → rollback` pattern. All other steps return a `Plan` object that drives a unified `describe → validate → execute` lifecycle. The `Plan` encapsulates operations and renders validation reports, operations summaries, and implications — the top-level dispatcher orchestrates all steps uniformly without knowing their internals.

Expand Down Expand Up @@ -54,6 +54,7 @@ Detailed documentation for subcommands is available in:
- [assess.md](./gen2-migration/assess.md) - Migration readiness assessment
- [generate.md](./gen2-migration/generate.md) - Code generation pipeline for transforming Gen1 configs to Gen2 TypeScript
- [refactor.md](./gen2-migration/refactor.md) - CloudFormation stack refactoring for moving stateful resources
- [retain.md](./gen2-migration/retain.md) - Apply retain policies below root so Gen1 can be deleted safely

## Architecture

Expand Down Expand Up @@ -133,16 +134,16 @@ amplify gen2-migration <step> [options]

### Subcommands

| Subcommand | Description | Implementation | Status |
| -------------- | --------------------------------------------------------------------- | ------------------------------------------------------- | --------------- |
| `assess` | Assess migration readiness for the Gen1 environment | `assess.ts` → `AmplifyMigrationAssessor` | Implemented |
| `clone` | Clone environment for migration | `clone.ts` → `AmplifyMigrationCloneStep` | NOT IMPLEMENTED |
| `lock` | Lock environment and enable deletion protection on stateful resources | `lock.ts` → `AmplifyMigrationLockStep` | Implemented |
| `generate` | Generate Gen2 backend code from Gen1 configuration | `generate.ts` → `AmplifyMigrationGenerateStep` | Implemented |
| `refactor` | Move stateful resources from Gen1 to Gen2 stacks | `refactor/refactor.ts` → `AmplifyMigrationRefactorStep` | Implemented |
| `shift` | Shift traffic to Gen2 | `shift.ts` → `AmplifyMigrationShiftStep` | NOT IMPLEMENTED |
| `decommission` | Delete Gen1 environment after migration | `decommission.ts` → `AmplifyMigrationDecommissionStep` | Implemented |
| `cleanup` | Clean up migration artifacts | `cleanup.ts` → `AmplifyMigrationCleanupStep` | NOT IMPLEMENTED |
| Subcommand | Description | Implementation | Status |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- | --------------- |
| `assess` | Assess migration readiness for the Gen1 environment | `assess.ts` → `AmplifyMigrationAssessor` | Implemented |
| `clone` | Clone environment for migration | `clone.ts` → `AmplifyMigrationCloneStep` | NOT IMPLEMENTED |
| `lock` | Lock environment, apply `DeletionPolicy: Retain` to every resource in every Gen1 stack, and enable DynamoDB deletion protection | `lock.ts` → `AmplifyMigrationLockStep` | Implemented |
| `generate` | Generate Gen2 backend code from Gen1 configuration | `generate.ts` → `AmplifyMigrationGenerateStep` | Implemented |
| `refactor` | Move stateful resources from Gen1 to Gen2 stacks | `refactor/refactor.ts` → `AmplifyMigrationRefactorStep` | Implemented |
| `retain` | Apply retain policies to every resource in every Gen1 stack below root | `retain.ts` → `AmplifyMigrationRetainStep` | Implemented |
| `shift` | Shift traffic to Gen2 | `shift.ts` → `AmplifyMigrationShiftStep` | NOT IMPLEMENTED |
| `cleanup` | Clean up migration artifacts | `cleanup.ts` → `AmplifyMigrationCleanupStep` | NOT IMPLEMENTED |

### Global Options

Expand All @@ -157,7 +158,7 @@ amplify gen2-migration <step> [options]

**Important considerations:**

- The step execution order matters: lock → generate → refactor → decommission. Each step validates prerequisites from previous steps.
- The step execution order matters: lock → generate → refactor → retain. Each step validates prerequisites from previous steps.
- The `clone`, `shift`, and `cleanup` steps are NOT IMPLEMENTED—they throw 'Method not implemented' errors.
- The `GEN2_MIGRATION_ENVIRONMENT_NAME` environment variable on the Amplify app tracks which environment is being migrated and prevents concurrent migrations.
- Stateful resources (defined in `STATEFUL_RESOURCES` set) require special handling—the module prevents their deletion and enables deletion protection.
Expand All @@ -179,6 +180,5 @@ amplify gen2-migration <step> [options]
- The `--skip-validations` flag bypasses safety checks—use with extreme caution in production.
- Environment mismatch between local and migration target will throw an error—ensure consistency.
- Rollback implementations are incomplete for most steps (throw 'Not Implemented' errors)—manual intervention may be needed on failure.
- The decommission step creates a changeset to analyze resources—this can timeout for large stacks.
- Cannot specify both `--rollback` and `--no-rollback` flags simultaneously.
- The lock step's rollback does not disable deletion protection on DynamoDB tables (preserves safety).
- The lock step's rollback removes the deny stack policy but does not undo retain policies or DynamoDB deletion protection (preserves safety).
88 changes: 88 additions & 0 deletions docs/packages/amplify-cli/src/commands/gen2-migration/retain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# retain

The retain subcommand applies `DeletionPolicy: Retain` and `UpdateReplacePolicy: Retain` to every resource in every Gen1 CloudFormation stack below the root. Once applied, the user can manually delete the Gen1 root stack and every underlying AWS resource (DynamoDB tables, S3 buckets, Cognito pools, AppSync APIs, Lambdas) survives as an orphan.

## Key Responsibilities

- Walks the Gen1 stack hierarchy pre-order (parent before children) starting from the root's children; root is excluded.
- For each stack, fetches the template lazily at execute time (not at plan time) and skips the CFN round-trip entirely when every resource already has retain.
- Applies retain to every resource **except** `AWS::CloudFormation::Stack` references. Leaving nested-stack references untouched keeps the parent changeset strictly additive on non-stack attributes and avoids forcing CFN to rewrite child `Properties`.
- Creates a CloudFormation changeset per stack and validates it against an allow list before executing.
- Rollback is not supported (`NotImplementedFault`). To undo, edit the CloudFormation templates directly.

## Architecture

```mermaid
flowchart TD
CLI["amplify gen2-migration retain"] --> STEP["AmplifyMigrationRetainStep"]
STEP --> WALK["walkStackHierarchy(rootStackId)"]
WALK -->|"pre-order DFS, excluding root"| IDS["stackIds[]"]
STEP --> CLASSIFY["classifyStacks()"]
CLASSIFY -->|"Map<stackId, DiscoveredResource>"| CTX["context"]
IDS --> BUILDOP["buildRetainOperation(stackId, resource)"]
CTX --> BUILDOP
BUILDOP -->|"per-stack AmplifyMigrationOperation"| PLAN["Plan"]
PLAN -->|"execute()"| EXEC["For each stack: fetchTemplate → mutate → createChangeSet → validate → executeChangeSet"]
```

### `AmplifyMigrationRetainStep`

[`src/commands/gen2-migration/retain.ts`](../../../../packages/amplify-cli/src/commands/gen2-migration/retain.ts)

Implements the standard step lifecycle. `forward()` builds a `Plan` of per-stack operations. `rollback()` throws `NotImplementedFault`.

### `walkStackHierarchy`

Recursive pre-order DFS over `AWS::CloudFormation::Stack` resource entries. Returns every stack in the tree except the root. Pre-order is required so parents are processed before children — any parent update triggers CFN's Automatic/Dynamic re-evaluation of nested stack references, which is benign when no `Properties` actually changed but clobbers children if retain is applied in a different order.

### `classifyStacks`

Builds `Map<stackId, DiscoveredResource>` that associates each nested stack with its Amplify `DiscoveredResource`. Used purely for UX: `Plan.describe` groups operations under `Resource: <category>/<name> (<service>)` headers, and the execute-time spinner carries matching labels.

For AppSync, the api-stack and every one of its nested children (per-model stacks, ConnectionStack, FunctionDirectiveStack, CustomResourcesjson) share the same api resource.

Stacks not classified fall through to the default `Project` group with stack-name-only labels.

### `buildRetainOperation`

Returns one `AmplifyMigrationOperation` per stack. The operation's `execute()` is lazy — it fetches the template, filters to non-`AWS::CloudFormation::Stack` resources, mutates their `DeletionPolicy` / `UpdateReplacePolicy` to `Retain`, creates the changeset, validates it via `isAllowedRetainChangeset`, and executes it.

Idempotent on reruns: if every target resource already has retain, the whole CFN round-trip is skipped. A second short-circuit handles the case where CFN's own "no changes" detection elides an edit (`Custom::*` resources with empty Properties — see [cloudformation-coverage-roadmap#1543](https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/1543)).

### `isAllowedRetainChangeset`

Allow-lists a retain-only changeset. Accepts two kinds of changes:

- Direct `DeletionPolicy` or `UpdateReplacePolicy` edits targeting `Retain`.
- CFN's own no-op Automatic/Dynamic re-evaluations on `AWS::CloudFormation::Stack` references, emitted on every parent update. These are bookkeeping — they don't trigger child reconciliation because no `Properties` value actually changed.

Any other change (Properties edits on non-stack resources, Add, Remove, Replacement=True) throws `MigrationError`.

## Design Notes

### Why skip the root stack

Updating root post-refactor risks cascading TemplateURL reconciliation through the entire tree. The root is left alone; the user manually deletes it after retain completes, and CFN cascades through the already-retained children.

### Why skip `AWS::CloudFormation::Stack` entries

Adding retain to a nested stack reference would be a no-op for child protection — the child's own retain state is what matters when the cascade delete hits it. Leaving the reference entry untouched keeps the parent changeset narrow (only non-stack attributes change) and keeps Plan output readable.

### Why lazy over eager

**Eager:** create all N changesets up front during `forward()` / plan time, then execute them sequentially. The problem — when any parent's changeset is executed, every child stack's pre-created changeset goes `OBSOLETE` because CFN marks pending changesets stale on any stack update. You end up re-creating most of the changesets anyway.

**Lazy:** defer changeset creation until each operation's `execute()` runs. Each stack's round-trip is `fetchTemplate → createChangeSet → executeChangeSet`, back-to-back, with no gap for the changeset to go stale. The template and parameters reflect the current deployed state at the moment we create the changeset.

Lazy wins because it avoids the OBSOLETE churn and keeps each operation self-contained.

### Why pre-order

Parent update must land before child update. If a child is retained first and the parent is updated next, CFN emits an Automatic/Dynamic re-evaluation on that child's reference. The re-evaluation is structurally benign (Target.Attribute=Properties, no actual value diff) but ordering matters for operator confidence — pre-order ensures the changeset inventory at each step is understandable.

## AI Development Notes

- The step runs after `lock`, `generate`, `refactor`, and user-side Gen2 sandbox validation in the e2e flow. At that point Gen1 stacks have drifted from their S3 `TemplateURL` (refactor moved resources without re-uploading child templates). Updating intermediates or root post-refactor is unsafe — the "skip `AWS::CloudFormation::Stack` entries in parent templates" rule is what makes updating intermediates safe here.
- Resources gated by a false `Condition` (for example the AppSync-generated `CustomResourcesjson` stack's `EmptyResource`, which has `Condition: AlwaysFalse`) are never deployed. A retain-only edit on such a resource produces an empty changeset — CFN returns "didn't contain changes" and the `!changeset` branch treats it as a no-op. The resource doesn't actually exist in the running stack, so there's nothing to retain. Purely cosmetic.
- To undo retain, run the retain templates through CloudFormation manually without the `DeletionPolicy`/`UpdateReplacePolicy` attributes. The step itself has no rollback path.
- When adding a new resource type to `KNOWN_RESOURCE_KEYS`, update `classifyStacks` — the exhaustive switch will force the compiler to flag it.
1 change: 0 additions & 1 deletion packages/amplify-cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,6 @@
"amplify-nodejs-function-runtime-provider": "2.5.33",
"amplify-python-function-runtime-provider": "2.4.55",
"aws-cdk-lib": "~2.189.1",
"bottleneck": "2.19.5",
"cdk-from-cfn": "^0.291.0",
"chalk": "^4.1.1",
"ci-info": "^3.8.0",
Expand Down
Loading
Loading