From 8b8e8dbe3d979dcd645485647556cb2bf8889034 Mon Sep 17 00:00:00 2001 From: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com> Date: Mon, 11 May 2026 12:08:59 -0400 Subject: [PATCH 1/2] graph storage spec Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com> --- .../checklists/requirements.md | 57 +++ .../contracts/app-graph-schema.yaml | 300 ++++++++++++ specs/003-git-app-graph-storage/data-model.md | 410 +++++++++++++++++ .../003-git-app-graph-storage/design-notes.md | 277 +++++++++++ specs/003-git-app-graph-storage/plan.md | 227 +++++++++ specs/003-git-app-graph-storage/quickstart.md | 268 +++++++++++ .../radius-graph-proposal.md | 98 ++++ specs/003-git-app-graph-storage/research.md | 239 ++++++++++ specs/003-git-app-graph-storage/spec.md | 430 ++++++++++++++++++ specs/003-git-app-graph-storage/tasks.md | 323 +++++++++++++ 10 files changed, 2629 insertions(+) create mode 100644 specs/003-git-app-graph-storage/checklists/requirements.md create mode 100644 specs/003-git-app-graph-storage/contracts/app-graph-schema.yaml create mode 100644 specs/003-git-app-graph-storage/data-model.md create mode 100644 specs/003-git-app-graph-storage/design-notes.md create mode 100644 specs/003-git-app-graph-storage/plan.md create mode 100644 specs/003-git-app-graph-storage/quickstart.md create mode 100644 specs/003-git-app-graph-storage/radius-graph-proposal.md create mode 100644 specs/003-git-app-graph-storage/research.md create mode 100644 specs/003-git-app-graph-storage/spec.md create mode 100644 specs/003-git-app-graph-storage/tasks.md diff --git a/specs/003-git-app-graph-storage/checklists/requirements.md b/specs/003-git-app-graph-storage/checklists/requirements.md new file mode 100644 index 0000000000..39ab93ebbd --- /dev/null +++ b/specs/003-git-app-graph-storage/checklists/requirements.md @@ -0,0 +1,57 @@ +# Specification Quality Checklist: Git App Graph Preview + +**Purpose**: Validate specification completeness and quality before proceeding to planning +**Created**: January 30, 2026 +**Last Validated**: January 30, 2026 +**Feature**: [spec.md](../spec.md) + +## Content Quality + +- [x] No implementation details (languages, frameworks, APIs) +- [x] Focused on user value and business needs +- [x] Written for non-technical stakeholders +- [x] All mandatory sections completed + +## Requirement Completeness + +- [x] No [NEEDS CLARIFICATION] markers remain +- [x] Requirements are testable and unambiguous +- [x] Success criteria are measurable +- [x] Success criteria are technology-agnostic (no implementation details) +- [x] All acceptance scenarios are defined +- [x] Edge cases are identified +- [x] Scope is clearly bounded +- [x] Dependencies and assumptions identified + +## Feature Readiness + +- [x] All functional requirements have clear acceptance criteria +- [x] User scenarios cover primary flows +- [x] Feature meets measurable outcomes defined in Success Criteria +- [x] No implementation details leak into specification + +## Notes + +**Validation Summary**: All items passed ✓ + +**Spec Statistics**: +- 6 user stories with clear priorities (P1: 2, P2: 2, P3: 2) +- 17 functional requirements (FR-001 to FR-017) +- 5 non-functional requirements (NFR-001 to NFR-005) +- 8 measurable success criteria (SC-001 to SC-008) +- 6 edge cases documented +- 5 open questions identified for planning phase + +**Notable Additions Since Initial Draft**: +- Problem Statement section added +- Constitution Alignment mapping added +- Testing Requirements section added (per Constitution Principle IV) +- Non-Functional Requirements added (organizational code quality standards) +- Cross-Repository Impact section added +- Open Questions section captures decisions needed + +**Minor Notes (Acceptable)**: +- NFRs reference Go/golangci-lint - acceptable as organizational standards, not implementation choices +- Cross-Repository Impact mentions repo paths - acceptable as planning information + +**Status**: Specification is ready for `/speckit.clarify` or `/speckit.plan` diff --git a/specs/003-git-app-graph-storage/contracts/app-graph-schema.yaml b/specs/003-git-app-graph-storage/contracts/app-graph-schema.yaml new file mode 100644 index 0000000000..15c04bf6e2 --- /dev/null +++ b/specs/003-git-app-graph-storage/contracts/app-graph-schema.yaml @@ -0,0 +1,300 @@ +openapi: 3.0.3 +info: + title: Radius App Graph Static API + description: | + Internal API contract for static app graph generation from Bicep files. + This contract defines the JSON schema for app graph output and diff operations. + + Note: This is a CLI-internal contract, not a REST API. The schema defines + the structure of JSON files written by `rad app graph` commands. + version: 1.0.0 + license: + name: Apache 2.0 + url: https://www.apache.org/licenses/LICENSE-2.0 + +paths: {} + +components: + schemas: + AppGraph: + type: object + description: Complete application topology extracted from Bicep files + required: + - metadata + - resources + - connections + properties: + metadata: + $ref: '#/components/schemas/AppGraphMetadata' + resources: + type: array + items: + $ref: '#/components/schemas/AppGraphResource' + connections: + type: array + items: + $ref: '#/components/schemas/AppGraphConnection' + + AppGraphMetadata: + type: object + description: Generation context and provenance information + required: + - generatedAt + - radiusCliVersion + - sourceFiles + - sourceHash + properties: + generatedAt: + type: string + format: date-time + description: UTC timestamp when this graph was generated + example: "2026-01-30T10:15:00Z" + radiusCliVersion: + type: string + description: Version of rad CLI used to generate this graph + example: "0.35.0" + sourceFiles: + type: array + items: + type: string + description: Bicep files that contributed to this graph + example: ["app.bicep", "modules/database.bicep"] + sourceHash: + type: string + pattern: "^sha256:[a-f0-9]{64}$" + description: SHA256 hash of source files for staleness detection + example: "sha256:7d865e959b2466918c9863afca942d0fb89d7c9ac0c99bafc3749504ded97730" + gitCommit: + type: string + description: Current git commit SHA (if in a git repository) + example: "abc123def456" + + AppGraphResource: + type: object + description: A single resource in the application topology + required: + - id + - name + - type + - sourceLocation + properties: + id: + type: string + description: Fully qualified Radius resource ID + pattern: "^/planes/radius/.+/providers/.+/.+" + example: "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend" + name: + type: string + description: Human-readable resource name + example: "frontend" + type: + type: string + description: Resource type + example: "Applications.Core/containers" + sourceLocation: + $ref: '#/components/schemas/SourceLocation' + gitInfo: + $ref: '#/components/schemas/GitInfo' + properties: + type: object + additionalProperties: true + description: Type-specific resource configuration + + SourceLocation: + type: object + description: Bicep source file and line where a resource is defined + required: + - file + - line + properties: + file: + type: string + description: Path to Bicep file (relative to repo root) + example: "app.bicep" + line: + type: integer + minimum: 1 + description: 1-based line number where resource begins + example: 12 + module: + type: string + description: Module path if from an imported module + example: "modules/database.bicep" + + GitInfo: + type: object + description: Git commit information for a resource + required: + - commitSha + - commitShort + - author + - date + - message + properties: + commitSha: + type: string + pattern: "^[a-f0-9]{40}$" + description: Full commit hash that last modified this resource + example: "abc123def456789012345678901234567890abcd" + commitShort: + type: string + pattern: "^[a-f0-9]{7}$" + description: Abbreviated commit hash for display + example: "abc123d" + author: + type: string + format: email + description: Commit author email + example: "dev@example.com" + date: + type: string + format: date-time + description: Commit timestamp in RFC3339 format + example: "2026-01-29T14:30:00Z" + message: + type: string + description: Commit message (first line only) + example: "Add frontend container" + uncommitted: + type: boolean + description: True if resource has uncommitted changes + default: false + + AppGraphConnection: + type: object + description: Directed edge between two resources + required: + - sourceId + - targetId + - type + properties: + sourceId: + type: string + description: Resource ID where connection originates + example: "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend" + targetId: + type: string + description: Resource ID where connection terminates + example: "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/backend" + type: + type: string + enum: + - connection + - route + - dependsOn + description: Kind of connection + + GraphDiff: + type: object + description: Differences between two app graphs + required: + - addedResources + - removedResources + - modifiedResources + - addedConnections + - removedConnections + - summary + properties: + baseCommit: + type: string + description: Commit SHA of the base graph + example: "abc123" + headCommit: + type: string + description: Commit SHA of the head graph + example: "def456" + addedResources: + type: array + items: + $ref: '#/components/schemas/AppGraphResource' + description: Resources in head but not in base + removedResources: + type: array + items: + $ref: '#/components/schemas/AppGraphResource' + description: Resources in base but not in head + modifiedResources: + type: array + items: + $ref: '#/components/schemas/ResourceDiff' + description: Resources with changed properties + addedConnections: + type: array + items: + $ref: '#/components/schemas/AppGraphConnection' + description: Connections in head but not in base + removedConnections: + type: array + items: + $ref: '#/components/schemas/AppGraphConnection' + description: Connections in base but not in head + summary: + $ref: '#/components/schemas/DiffSummary' + + ResourceDiff: + type: object + description: Changes to a single resource + required: + - id + - name + - type + - changedProperties + properties: + id: + type: string + description: Resource ID + name: + type: string + description: Resource name + type: + type: string + description: Resource type + changedProperties: + type: array + items: + $ref: '#/components/schemas/PropertyChange' + + PropertyChange: + type: object + description: Single property modification + required: + - path + properties: + path: + type: string + description: JSON path to changed property + example: "properties.container.image" + oldValue: + description: Value in base graph + newValue: + description: Value in head graph + + DiffSummary: + type: object + description: Aggregate statistics for the diff + required: + - totalChanges + - resourcesAdded + - resourcesRemoved + - resourcesModified + - connectionsAdded + - connectionsRemoved + properties: + totalChanges: + type: integer + minimum: 0 + resourcesAdded: + type: integer + minimum: 0 + resourcesRemoved: + type: integer + minimum: 0 + resourcesModified: + type: integer + minimum: 0 + connectionsAdded: + type: integer + minimum: 0 + connectionsRemoved: + type: integer + minimum: 0 diff --git a/specs/003-git-app-graph-storage/data-model.md b/specs/003-git-app-graph-storage/data-model.md new file mode 100644 index 0000000000..cc35aac69a --- /dev/null +++ b/specs/003-git-app-graph-storage/data-model.md @@ -0,0 +1,410 @@ +# Data Model: Git App Graph Preview + +**Feature Branch**: `001-git-app-graph-preview` +**Date**: February 4, 2026 + +## Core Entities + +### AppGraph + +The root container for all graph data. This is the primary structure serialized to JSON. + +```go +// AppGraph represents the complete application topology extracted from Bicep files. +// This is the primary output of the `rad app graph ` command. +type AppGraph struct { + // Metadata contains generation context and provenance information + Metadata AppGraphMetadata `json:"metadata"` + + // Resources contains all resource nodes in the application + Resources []AppGraphResource `json:"resources"` + + // Connections contains all edges between resources + Connections []AppGraphConnection `json:"connections"` +} +``` + +### AppGraphMetadata + +Provenance and staleness detection information. + +```go +// AppGraphMetadata contains generation context for the app graph. +type AppGraphMetadata struct { + // GeneratedAt is the UTC timestamp when this graph was generated + GeneratedAt time.Time `json:"generatedAt"` + + // RadiusCliVersion is the version of rad CLI used to generate this graph + RadiusCliVersion string `json:"radiusCliVersion"` + + // SourceFiles lists all Bicep files that contributed to this graph + SourceFiles []string `json:"sourceFiles"` + + // SourceHash is a SHA256 hash of all source files for staleness detection + SourceHash string `json:"sourceHash"` + + // GitCommit is the current git commit SHA (if in a git repository) + GitCommit string `json:"gitCommit,omitempty"` +} +``` + +### AppGraphResource + +A single resource node in the application graph. + +```go +// AppGraphResource represents a single resource in the application topology. +type AppGraphResource struct { + // ID is the fully qualified Radius resource ID + // Format: /planes/radius/local/resourceGroups/{rg}/providers/{type}/{name} + ID string `json:"id"` + + // Name is the human-readable resource name + Name string `json:"name"` + + // Type is the resource type (e.g., Applications.Core/containers) + Type string `json:"type"` + + // SourceLocation indicates where this resource is defined + SourceLocation SourceLocation `json:"sourceLocation"` + + // GitInfo contains git metadata for this resource (optional) + GitInfo *GitInfo `json:"gitInfo,omitempty"` + + // Properties contains type-specific resource configuration + // Stored as map for flexibility across resource types + Properties map[string]any `json:"properties,omitempty"` +} +``` + +### SourceLocation + +Source file tracking for each resource. + +```go +// SourceLocation indicates the Bicep source file and line where a resource is defined. +type SourceLocation struct { + // File is the path to the Bicep file (relative to repo root) + File string `json:"file"` + + // Line is the 1-based line number where the resource begins + Line int `json:"line"` + + // Module is the module path if this resource is from an imported module + Module string `json:"module,omitempty"` +} +``` + +### GitInfo + +Git metadata for a resource, populated via `git blame`. + +```go +// GitInfo contains git commit information for a resource. +type GitInfo struct { + // CommitSHA is the full commit hash that last modified this resource + CommitSHA string `json:"commitSha"` + + // CommitShort is the abbreviated commit hash for display + CommitShort string `json:"commitShort"` + + // Author is the commit author email + Author string `json:"author"` + + // Date is the commit timestamp in RFC3339 format + Date time.Time `json:"date"` + + // Message is the commit message (first line only) + Message string `json:"message"` + + // Uncommitted indicates this resource has uncommitted changes + Uncommitted bool `json:"uncommitted,omitempty"` +} +``` + +### AppGraphConnection + +An edge between two resources representing a dependency or data flow. + +```go +// AppGraphConnection represents a directed edge between two resources. +type AppGraphConnection struct { + // SourceID is the resource ID where the connection originates + SourceID string `json:"sourceId"` + + // TargetID is the resource ID where the connection terminates + TargetID string `json:"targetId"` + + // Type indicates the kind of connection + Type ConnectionType `json:"type"` +} + +// ConnectionType enumerates the kinds of resource connections. +type ConnectionType string + +const ( + // ConnectionTypeConnection represents a direct connection (e.g., container to database) + ConnectionTypeConnection ConnectionType = "connection" + + // ConnectionTypeRoute represents a gateway route to a destination + ConnectionTypeRoute ConnectionType = "route" + + // ConnectionTypeDependsOn represents an explicit dependsOn relationship + ConnectionTypeDependsOn ConnectionType = "dependsOn" +) +``` + +## Diff Entities + +### GraphDiff + +The result of comparing two app graphs. + +```go +// GraphDiff represents the differences between two app graphs. +type GraphDiff struct { + // BaseCommit is the commit SHA of the base graph (optional) + BaseCommit string `json:"baseCommit,omitempty"` + + // HeadCommit is the commit SHA of the head graph (optional) + HeadCommit string `json:"headCommit,omitempty"` + + // AddedResources are resources present in head but not in base + AddedResources []AppGraphResource `json:"addedResources"` + + // RemovedResources are resources present in base but not in head + RemovedResources []AppGraphResource `json:"removedResources"` + + // ModifiedResources are resources with changed properties + ModifiedResources []ResourceDiff `json:"modifiedResources"` + + // AddedConnections are connections present in head but not in base + AddedConnections []AppGraphConnection `json:"addedConnections"` + + // RemovedConnections are connections present in base but not in head + RemovedConnections []AppGraphConnection `json:"removedConnections"` + + // Summary provides a human-readable overview + Summary DiffSummary `json:"summary"` +} + +// ResourceDiff captures changes to a single resource. +type ResourceDiff struct { + // ID is the resource ID (same in both base and head) + ID string `json:"id"` + + // Name is the resource name + Name string `json:"name"` + + // Type is the resource type + Type string `json:"type"` + + // ChangedProperties lists the property paths that changed + ChangedProperties []PropertyChange `json:"changedProperties"` +} + +// PropertyChange describes a single property modification. +type PropertyChange struct { + // Path is the JSON path to the changed property (e.g., "properties.container.image") + Path string `json:"path"` + + // OldValue is the value in the base graph + OldValue any `json:"oldValue,omitempty"` + + // NewValue is the value in the head graph + NewValue any `json:"newValue,omitempty"` +} + +// DiffSummary provides aggregate statistics for the diff. +type DiffSummary struct { + TotalChanges int `json:"totalChanges"` + ResourcesAdded int `json:"resourcesAdded"` + ResourcesRemoved int `json:"resourcesRemoved"` + ResourcesModified int `json:"resourcesModified"` + ConnectionsAdded int `json:"connectionsAdded"` + ConnectionsRemoved int `json:"connectionsRemoved"` +} +``` + +## Entity Relationships + +``` +┌─────────────────┐ +│ AppGraph │ +├─────────────────┤ +│ Metadata │──────┐ +│ Resources[] │──┐ │ +│ Connections[] │ │ │ +└─────────────────┘ │ │ + │ │ + ┌───────────────┘ │ + ▼ ▼ +┌─────────────────┐ ┌──────────────────┐ +│ AppGraphResource│ │ AppGraphMetadata │ +├─────────────────┤ ├──────────────────┤ +│ ID │ │ GeneratedAt │ +│ Name │ │ RadiusCliVersion │ +│ Type │ │ SourceFiles[] │ +│ SourceLocation │──┐ SourceHash │ +│ GitInfo? │ │ GitCommit? │ +│ Properties │ └──────────────────┘ +└─────────────────┘ + │ + ┌────┴────┐ + ▼ ▼ +┌──────────┐ ┌─────────┐ +│SourceLoc │ │ GitInfo │ +├──────────┤ ├─────────┤ +│ File │ │CommitSHA│ +│ Line │ │Author │ +│ Module? │ │Date │ +└──────────┘ │Message │ + └─────────┘ + +┌────────────────────┐ +│ AppGraphConnection │ +├────────────────────┤ +│ SourceID │───────► AppGraphResource.ID +│ TargetID │───────► AppGraphResource.ID +│ Type │ +└────────────────────┘ +``` + +## Validation Rules + +### AppGraph +- `Metadata.GeneratedAt` MUST be a valid UTC timestamp +- `Metadata.SourceFiles` MUST contain at least one file +- `Metadata.SourceHash` MUST be a valid SHA256 hash (64 hex chars) +- `Resources` MAY be empty for an empty Bicep file + +### AppGraphResource +- `ID` MUST be a valid Radius resource ID format +- `Name` MUST be non-empty +- `Type` MUST be a recognized resource type or follow ARM type pattern +- `SourceLocation.File` MUST be a valid relative file path +- `SourceLocation.Line` MUST be >= 1 + +### AppGraphConnection +- `SourceID` MUST reference an existing resource ID +- `TargetID` MUST reference an existing resource ID +- `Type` MUST be one of the defined ConnectionType values + +### GraphDiff +- All resource references in Added/Removed/Modified MUST be valid +- `Summary` counts MUST match the actual array lengths + +## State Transitions + +Resources can be in the following states relative to git: + +``` +┌──────────────┐ +│ Uncommitted │ ───(git add)───► ┌──────────┐ +└──────────────┘ │ Staged │ + └──────────┘ + │ + (git commit) + │ + ▼ +┌──────────────┐ ┌──────────┐ +│ Modified │ ◄──(edit file)──│Committed │ +└──────────────┘ └──────────┘ + │ + (git add + commit) + │ + ▼ + ┌──────────┐ + │Committed │ (new SHA) + └──────────┘ +``` + +Graph states: +- **Current**: Generated from current working directory files +- **Committed**: Exists in `.radius/app-graph.json` in git history +- **Stale**: Committed graph doesn't match current Bicep files (detected via sourceHash) + +## JSON Schema Example + +```json +{ + "metadata": { + "generatedAt": "2026-01-30T10:15:00Z", + "radiusCliVersion": "0.35.0", + "sourceFiles": [ + "app.bicep", + "modules/database.bicep" + ], + "sourceHash": "sha256:7d865e959b2466918c9863afca942d0fb89d7c9ac0c99bafc3749504ded97730", + "gitCommit": "abc123def456" + }, + "resources": [ + { + "id": "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend", + "name": "frontend", + "type": "Applications.Core/containers", + "sourceLocation": { + "file": "app.bicep", + "line": 12 + }, + "gitInfo": { + "commitSha": "abc123def456789012345678901234567890abcd", + "commitShort": "abc123d", + "author": "dev@example.com", + "date": "2026-01-29T14:30:00Z", + "message": "Add frontend container" + }, + "properties": { + "container": { + "image": "myapp/frontend:v1.2.3" + } + } + }, + { + "id": "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/backend", + "name": "backend", + "type": "Applications.Core/containers", + "sourceLocation": { + "file": "app.bicep", + "line": 28 + }, + "gitInfo": { + "commitSha": "def456abc789012345678901234567890abcdef01", + "commitShort": "def456a", + "author": "dev@example.com", + "date": "2026-01-28T09:15:00Z", + "message": "Add backend service" + } + }, + { + "id": "/planes/radius/local/resourceGroups/default/providers/Applications.Datastores/redisCaches/cache", + "name": "cache", + "type": "Applications.Datastores/redisCaches", + "sourceLocation": { + "file": "modules/database.bicep", + "line": 5, + "module": "modules/database.bicep" + }, + "gitInfo": { + "commitSha": "789abc012def345678901234567890abcdef0123", + "commitShort": "789abc0", + "author": "ops@example.com", + "date": "2026-01-27T16:45:00Z", + "message": "Add Redis cache for session storage" + } + } + ], + "connections": [ + { + "sourceId": "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend", + "targetId": "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/backend", + "type": "connection" + }, + { + "sourceId": "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/backend", + "targetId": "/planes/radius/local/resourceGroups/default/providers/Applications.Datastores/redisCaches/cache", + "type": "connection" + } + ] +} +``` diff --git a/specs/003-git-app-graph-storage/design-notes.md b/specs/003-git-app-graph-storage/design-notes.md new file mode 100644 index 0000000000..09f0d9ea73 --- /dev/null +++ b/specs/003-git-app-graph-storage/design-notes.md @@ -0,0 +1,277 @@ +# Design Notes: Git App Graph Preview + +**Feature Branch**: `001-git-app-graph-preview` +**Last Updated**: February 5, 2026 +**Related**: [spec.md](spec.md) + +--- + +## Conceptual Framework + +### The Graph as a File Projection + +A fundamental aspect of the app graph is that it serves as a **projection of the files associated with an application**. Rather than being a standalone artifact, the graph is computed from and represents the source files that define the application's structure. + +### Runtime Environments + +The graph is designed to operate in multiple contexts: + +| Environment | Description | Use Case | +| --- | --- | --- | +| **Local workstation** | Graph generated from a local clone of a repository | Developer preview, local validation, pre-commit checks | +| **Without git** | Graph generated from standalone files (no git repository) | Quick prototyping, isolated testing, non-versioned projects | +| **Control plane server** | Graph managed by Radius UCP or similar orchestrator | Enterprise scenarios, centralized graph management, cross-repo analysis | +| **GitHub (CI/CD)** | Graph generated or read within GitHub Actions | PR visualization, automated validation, deployment pipelines | + +This flexibility ensures the graph works across the full development lifecycle—from local experimentation through production deployment—without requiring a specific hosting model. + +### Serialization and Multi-User Considerations + +The graph's storage format varies by runtime environment, with different trade-offs for collaboration: + +| Environment | Storage | Conflict Handling | +| --- | --- | --- | +| **Local/Git** | JSON file (`.radius/app-graph.json`) | Git branching and merging | +| **Server-hosted** | Graph database | Transactional updates, no merge conflicts | + +## Local File Format Design Goals + +For users without a server instance, the serialized graph file must be as friendly as possible to branching and merging: + +- **Deterministic ordering**: Resources and connections sorted alphabetically by ID, ensuring identical inputs produce identical outputs across branches +- **One resource per logical block**: JSON structure organized so that adding/removing a resource affects minimal lines, reducing merge conflict surface area +- **Stable identifiers**: Resource IDs derived from source content rather than generation order, so the same resource in different branches has the same ID +- **Human-readable format**: Pretty-printed JSON with consistent indentation, making manual conflict resolution feasible when needed + +## Server-Hosted Graph Database + +When the graph is hosted on a control plane server, a proper graph database (e.g., Neo4j, Amazon Neptune, or an embedded solution) can be used: + +- **Transactional updates**: Multiple users can modify the graph concurrently without file-level conflicts +- **Query capabilities**: Rich traversal and pattern matching for cross-repository analysis +- **Real-time collaboration**: Changes propagate immediately without commit/push cycles +- **Historical versioning**: Built-in temporal queries for graph state at any point in time + +The graph schema remains consistent across both storage backends—only the serialization and conflict resolution mechanisms differ. + +### Standard Schema for Universal Representation + +The graph would have a **standard schema** that allows it to represent the app graph. While it could be extended to represent other deployment-related concepts, the app graph is the core component of this schema. + +This schema-first approach enables: + +- Consistent tooling regardless of source format +- Interoperability between different deployment technologies +- A stable foundation for visualization and diffing + +### Multi-Format Source Support + +Today, the app model is represented in an `app.bicep` file for Radius users. However, the app graph is designed to work with **any application representation format**: + +- Radius `app.bicep` files +- Kustomize files +- GitOps repository configurations +- KRO (Kubernetes Resource Orchestrator) files +- Kubernetes CRDs +- Other declarative infrastructure formats + +### Schema Mapping Architecture + +The graph understands different source formats through a **two-layer approach**: + +1. **Standard Schema**: A universal schema that defines how applications are represented in the graph (resources, connections, metadata) + +2. **Schema Mappings**: Format-specific mappings that translate from each source type to the standard graph schema: + - `radius` → graph (out-of-the-box, first-class support) + - `kustomize` → graph + - `kubernetes-crds` → graph + - `kro` → graph + - etc. + +The Radius schema mapping would be provided out of the box, with mappings for other formats as an extensibility point. + +### Lighting Up Experiences in Any Repository + +This architecture allows Radius to **light up experiences in any repository**: + +- Add Radius to a GitOps pipeline, and the graph understands the GitOps configurations +- The graph loads itself from whatever source format exists +- Visualization, diffing, and PR integration work regardless of the underlying technology + +### Recipe Registration Against the Graph + +If the app graph is loaded (from any source format), we can then **register recipes against the app graph**. This decouples recipe registration from the source format, allowing Radius deployments to work with any technology. + +```text +┌─────────────────────────────────────────────────────────────────────────────┐ +│ SOURCE FORMATS │ +├─────────────┬─────────────┬─────────────┬─────────────┬─────────────────────┤ +│ app.bicep │ kustomize │ kro files │ k8s CRDs │ gitops configs │ +└──────┬──────┴──────┬──────┴──────┬──────┴──────┬──────┴──────────┬──────────┘ + │ │ │ │ │ + ▼ ▼ ▼ ▼ ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ SCHEMA MAPPINGS │ +│ (format-specific parsers that translate to standard graph schema) │ +└─────────────────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ APP GRAPH │ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│ │ Foo │──▶│ Bar │───▶│ Baz │ │ +│ └─────────────┘ └─────────────┘ └─────────────┘ │ +│ (standard schema representation) │ +└─────────────────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ RECIPE REGISTRATION │ +│ Recipes bound to types in the graph │ +└─────────────────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ DEPLOYMENT │ +│ Graph + Recipes → Concrete infrastructure in target environment │ +└─────────────────────────────────────────────────────────────────────────────┘ +``` + +### Migration Path: GitOps to Bicep + +The graph facilitates **migration between deployment technologies**. For example, migrating from a GitOps workflow to a simpler `app.bicep` representation: + +```text +┌─────────────────────────────────────────────────────────────────────────────┐ +│ GITOPS REPOSITORY │ +│ ├── clusters/ │ +│ │ └── prod/ │ +│ │ ├── flux-system/ │ +│ │ └── apps/ │ +│ │ ├── frontend-deployment.yaml │ +│ │ ├── backend-deployment.yaml │ +│ │ ├── redis-statefulset.yaml │ +│ │ └── kustomization.yaml │ +│ └── base/ │ +│ └── ... │ +└─────────────────────────────────┬───────────────────────────────────────────┘ + │ + │ (1) Parse GitOps configs + │ via kustomize schema mapping + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ APP GRAPH │ +│ │ +│ Resources: Connections: │ +│ ┌──────────────────┐ ┌──────────────────────────────────┐ │ +│ │ frontend │ │ frontend → backend (http:8080) │ │ +│ │ type: Container │ │ backend → redis (tcp:6379) │ │ +│ │ image: nginx:1.2 │ └──────────────────────────────────┘ │ +│ ├──────────────────┤ │ +│ │ backend │ Metadata: │ +│ │ type: Container │ • sourceFormat: kustomize │ +│ │ image: api:3.1 │ • migrationDate: 2026-02-05 │ +│ ├──────────────────┤ • originalFiles: [list of yamls] │ +│ │ redis │ │ +│ │ type: Cache │ │ +│ └──────────────────┘ │ +└─────────────────────────────────┬───────────────────────────────────────────┘ + │ + │ (2) Serialize graph to Bicep + │ via radius schema mapping + ▼ +┌─────────────────────────────────────────────────────────────────────────────┐ +│ app.bicep │ +│ │ +│ resource frontend 'Applications.Core/containers@2023-10-01' = { │ +│ name: 'frontend' │ +│ properties: { │ +│ container: { image: 'nginx:1.2' } │ +│ connections: { backend: { source: backend.id } } │ +│ } │ +│ } │ +│ │ +│ resource backend 'Applications.Core/containers@2023-10-01' = { ... } │ +│ resource redis 'Applications.Datastores/redisCaches@2023-10-01' = { ... } │ +└─────────────────────────────────────────────────────────────────────────────┘ + │ + │ (3) Validate equivalence + │ regenerate graph from bicep, + ▼ compare to original + + ✓ Migration Complete +``` + +Because both formats map to the same intermediate graph representation, translation becomes straightforward: + +1. Parse GitOps configs into the graph +2. Serialize the graph to `app.bicep` format +3. Validate equivalence + +### Intelligent State (vs. Terraform State) + +Another way of thinking about the graph is that it represents **intelligent state**—similar in purpose to Terraform state, but without the associated management problems: + +| Terraform State | App Graph | +| ----------------- | ----------- | +| Stores resource mappings and metadata | Stores resource relationships and metadata | +| Requires remote backends, locking, conflict resolution | Committed to git, versioned naturally | +| State drift causes deployment failures | Graph is regenerated from source, no drift | +| Sensitive data in state files | No runtime secrets—purely structural | +| Single source of truth (fragile) | Derived artifact (resilient) | + +The app graph is a **derived view** rather than a primary data store, which eliminates the operational burden of state management while providing the benefits of understanding what was deployed and how it relates. + +--- + +## Implications for Implementation + +These conceptual foundations inform several implementation decisions in the [spec](spec.md): + +1. **Committed Artifact Model**: The graph is committed to version control (`.radius/app-graph.json`) because it's a derived projection, not primary state + +2. **Deterministic Output**: Same input must produce identical output to support git-based versioning + +3. **Schema-First Design**: The JSON graph format should be designed for extensibility to support future schema mappings + +4. **Lightweight GitHub Action**: The Action reads committed JSON rather than generating graphs, keeping it source-format agnostic + +5. **Staleness Detection**: The `sourceHash` field enables validation that the graph matches its source files + +## Appendix 1: Radius Conceptual Graph + +```mermaid +flowchart TD + Workspace["Workspace"] + Group["Group"] + Environment["Environment"] + RecipePack["Recipe Pack"] + Recipe["Recipe"] + Deployment["Deployment"] + ResourceType["Resource Type"] + DeployedResource["Deployed Resource
(Resource Instance)"] + Workspace --> Environment + Workspace --> Group + + %% Environment relationships + Environment --> RecipePack + Environment --> Deployment + + %% Deployment selects group + Group --> Deployment + + %% Recipes and resource types + RecipePack --> Recipe + Recipe --> ResourceType + + %% Deployment uses recipes and types + Deployment --> ResourceType + Deployment --> Recipe + + %% Deployment produces resource instances + Deployment --> DeployedResource + + %% Resource instances are of a type and produced by a recipe + DeployedResource --> ResourceType + DeployedResource --> Recipe +``` \ No newline at end of file diff --git a/specs/003-git-app-graph-storage/plan.md b/specs/003-git-app-graph-storage/plan.md new file mode 100644 index 0000000000..590b591ee5 --- /dev/null +++ b/specs/003-git-app-graph-storage/plan.md @@ -0,0 +1,227 @@ +# Implementation Plan: Git App Graph Preview + +**Branch**: `001-git-app-graph-preview` | **Date**: February 4, 2026 | **Spec**: [spec.md](./spec.md) +**Input**: Feature specification from `/specs/001-git-app-graph-preview/spec.md` +**User Prompt**: /speckit.plan Leverage the existing Radius technology stack for CLI, etc. as well as the application and environment, and resource definitions (https://github.com/radius-project/radius, https://docs.radapp.io/). Data structures and general diff constructs should use Git principles while specific integrations and renderings in the GitHub UI should use their technologies (e.g. GitHub Actions, GitHub Apps, etc.) such that if we were to build similar integrations in the future (for example, with GitLab), our design choices today won't limit us in the future. + +## Summary + +Enable **static app graph generation** from Bicep files without deployment, enriched with **git changelog metadata**, to support visualization and diffing in GitHub PR workflows. The implementation extends the existing `rad app graph` command to accept Bicep file input, generates deterministic JSON output suitable for version control, and provides a lightweight GitHub Action for PR diff visualization. + +**Technical Approach** (from [research.md](./research.md)): +- Use official Bicep CLI to compile `.bicep` → ARM JSON, then extract resources and connections +- Extend existing CLI patterns in `pkg/cli/cmd/app/graph/` with file-based input detection +- Add git metadata via shell exec (`git blame`, `git log`) for source tracking +- Output to `.radius/app-graph.json` (committed artifact model) +- GitHub Action reads JSON from git history, computes diff, posts Mermaid-enhanced PR comments + +## Technical Context + +**Language/Version**: Go 1.21+ (matches existing Radius codebase) +**Primary Dependencies**: +- Cobra (CLI framework, existing) +- Bicep CLI (external, managed via `rad bicep download`) +- Git (external, system requirement) +**Storage**: File-based (`.radius/app-graph.json` committed to git) +**Testing**: `make test` (Go unit tests), `make functional-test` (E2E) +**Target Platform**: Linux, macOS, Windows (cross-platform CLI) +**Project Type**: CLI extension (single project, existing radius repository) +**Performance Goals**: +- Graph generation < 5s for 50 resources (per SC-001) +- Git enrichment adds < 2s for 1000 commits (per SC-006) +**Constraints**: +- Must work without Radius environment (static analysis only) +- Must handle Bicep files up to 5000 lines (per SC-007) +**Scale/Scope**: Applications with up to 100+ resources, graphs committed across all Radius users + +## Constitution Check + +*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.* + +| Principle | Status | Notes | +|-----------|--------|-------| +| **I. API-First Design** | ✅ PASS | JSON schema defined in [contracts/](./contracts/), OpenAPI spec provided | +| **II. Idiomatic Code Standards** | ✅ PASS | Go implementation follows Effective Go; uses existing `pkg/cli/` patterns | +| **III. Multi-Cloud Neutrality** | ✅ PASS | Works with any cloud's Bicep resources; cloud-specific types rendered generically | +| **IV. Testing Pyramid Discipline** | ✅ PASS | Unit tests for parsing/diffing, integration tests for git/Bicep CLI, functional E2E tests | +| **V. Collaboration-Centric Design** | ✅ PASS | Developers preview graphs locally; platform engineers review in PRs | +| **VI. Open Source & Community-First** | ✅ PASS | Design spec in public repo; GitHub Action works in forks | +| **VII. Simplicity Over Cleverness** | ✅ PASS | Uses Bicep CLI (not custom parser); shell exec for git (not libgit2) | +| **VIII. Separation of Concerns** | ✅ PASS | Core diff logic platform-agnostic; GitHub rendering layer separate | +| **IX. Incremental Adoption** | ✅ PASS | New capability; doesn't change existing `rad app graph ` behavior | +| **XIV. Documentation Quality** | ✅ PASS | [quickstart.md](./quickstart.md) follows Diátaxis tutorial pattern | +| **XVII. Polyglot Coherence** | ✅ PASS | Cross-repo impact documented in spec; coordinated with `radius` and `docs` repos | + +**Post-Design Re-Check**: All principles remain satisfied. Platform abstraction in architecture supports future GitLab integration per user requirement. + +## Project Structure + +### Documentation (this feature) + +```text +specs/001-git-app-graph-preview/ +├── spec.md # Feature specification (input) +├── plan.md # This file +├── research.md # Phase 0 output (technology decisions) +├── data-model.md # Phase 1 output (entity definitions) +├── quickstart.md # Phase 1 output (user guide) +├── contracts/ # Phase 1 output (JSON schema) +│ └── app-graph-schema.yaml +└── tasks.md # Phase 2 output (implementation tasks) +``` + +### Source Code (radius repository) + +```text +# radius repository (../radius) +pkg/ +├── cli/ +│ ├── cmd/ +│ │ └── app/ +│ │ └── graph/ +│ │ ├── graph.go # Extended: file input detection +│ │ ├── static.go # NEW: static graph generation +│ │ ├── diff.go # NEW: graph diff computation +│ │ ├── display.go # Extended: Markdown/Mermaid output +│ │ └── graph_test.go # Extended: new test cases +│ ├── bicep/ +│ │ └── parser.go # NEW: ARM JSON → AppGraph extraction +│ └── git/ +│ └── metadata.go # NEW: git blame/log integration +├── corerp/ +│ └── api/ +│ └── v20231001preview/ +│ └── appgraph_types.go # Extended: new types for static graphs + +test/ +├── unit/ +│ └── cli/ +│ └── graph/ # NEW: unit tests for graph generation +├── integration/ +│ └── cli/ +│ └── graph/ # NEW: integration tests with git/Bicep +└── functional/ + └── cli/ + └── graph/ # NEW: E2E test scenarios + +# GitHub Action (separate repo or actions/ folder) +actions/ +└── app-graph-diff/ + ├── action.yml # Action definition + ├── diff.sh # Diff computation script + └── render.js # Markdown/Mermaid rendering +``` + +**Structure Decision**: Extends existing `radius` repository structure. New code follows established patterns in `pkg/cli/`. GitHub Action is a separate deliverable, potentially in its own repo (`radius-project/app-graph-diff-action`). + +## Architecture + +### Component Diagram + +``` +┌─────────────────────────────────────────────────────────────────────────┐ +│ rad CLI │ +├─────────────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ +│ │ graph.go │ │ static.go │ │ diff.go │ │ +│ │ (entry point) │───▶│ (file input) │───▶│ (comparison) │ │ +│ │ │ │ │ │ │ │ +│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ +│ │ │ │ │ +│ ▼ ▼ ▼ │ +│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ +│ │ Radius API │ │ Bicep CLI │ │ Git CLI │ │ +│ │ (deployed apps) │ │ (compilation) │ │ (metadata) │ │ +│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ +│ │ │ │ +│ ▼ ▼ │ +│ ┌──────────────────┐ ┌──────────────────┐ │ +│ │ parser.go │ │ metadata.go │ │ +│ │ (ARM→AppGraph) │ │ (blame/log) │ │ +│ └──────────────────┘ └──────────────────┘ │ +│ │ │ │ +│ └───────────┬───────────┘ │ +│ ▼ │ +│ ┌──────────────────────┐ │ +│ │ AppGraph (JSON) │ │ +│ │ .radius/app-graph.json │ +│ └──────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────┐ +│ GitHub Action │ +├─────────────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ +│ │ Checkout │───▶│ diff.sh │───▶│ render.js │ │ +│ │ (base + head) │ │ (JSON compare) │ │ (Mermaid) │ │ +│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌──────────────────────┐ │ +│ │ PR Comment │ │ +│ │ (create-or-update) │ │ +│ └──────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────┘ +``` + +### Platform Abstraction (per user requirement) + +``` +┌─────────────────────────────────────────────────────────────────────────┐ +│ Platform-Agnostic Core │ +├─────────────────────────────────────────────────────────────────────────┤ +│ • AppGraph data model (JSON schema) │ +│ • Bicep → ARM → AppGraph parsing │ +│ • Git metadata extraction │ +│ • Diff computation (JSON semantic comparison) │ +│ • Core rendering (Markdown tables, Mermaid diagrams) │ +└─────────────────────────────────────────────────────────────────────────┘ + │ + ┌─────────────────────────┼─────────────────────────┐ + ▼ ▼ ▼ +┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ +│ GitHub Layer │ │ GitLab Layer │ │ CLI Layer │ +│ (Action) │ │ (Future) │ │ (Terminal) │ +├──────────────────┤ ├──────────────────┤ ├──────────────────┤ +│ • PR comments │ │ • MR notes │ │ • stdout │ +│ • Workflow YAML │ │ • CI YAML │ │ • --format │ +│ • Mermaid render│ │ • Mermaid render│ │ • diff command │ +└──────────────────┘ └──────────────────┘ └──────────────────┘ +``` + +## Implementation Phases + +### Phase 1 (P1) - Core Graph Generation +- US1: Generate app graph from Bicep files +- US2: Export graph as diff-friendly JSON/Markdown + +### Phase 2 (P2) - Git & GitHub Integration +- US3: Git metadata enrichment +- US4: GitHub Action for PR diff comments + +### Phase 3 (P3) - Advanced Features +- US5: Historical graph timeline +- US6: Environment-resolved graphs + +## Complexity Tracking + +> No complexity violations identified. + +| Principle | Evaluation | Status | +|-----------|------------|--------| +| Simplicity Over Cleverness | Using Bicep CLI (not custom parser) | ✅ Simple | +| Simplicity Over Cleverness | Shell exec for git (not libgit2) | ✅ Simple | +| Incremental Adoption | Additive to existing CLI; no breaking changes | ✅ Non-disruptive | + +## Generated Artifacts + +- [research.md](./research.md) - Technology decisions and alternatives +- [data-model.md](./data-model.md) - Entity definitions and relationships +- [quickstart.md](./quickstart.md) - User-facing tutorial +- [contracts/app-graph-schema.yaml](./contracts/app-graph-schema.yaml) - OpenAPI JSON schema + +## Next Steps + +Run `/speckit.tasks` to generate actionable implementation tasks from this plan. diff --git a/specs/003-git-app-graph-storage/quickstart.md b/specs/003-git-app-graph-storage/quickstart.md new file mode 100644 index 0000000000..90bb3f65ff --- /dev/null +++ b/specs/003-git-app-graph-storage/quickstart.md @@ -0,0 +1,268 @@ +# Quickstart: Git App Graph Preview + +This guide walks you through generating and using app graphs from Bicep files. + +## Prerequisites + +- [Radius CLI](https://docs.radapp.io/getting-started/install/) (v0.35.0+) +- [Bicep CLI](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/install) (installed automatically by `rad bicep download`) +- A Bicep file defining a Radius application +- Git repository (optional, for git metadata enrichment) + +## 1. Generate an App Graph + +Given a Bicep file `app.bicep`: + +```bicep +extension radius + +@description('The application name') +param appName string = 'myapp' + +resource app 'Applications.Core/applications@2023-10-01-preview' = { + name: appName + properties: { + environment: environment + } +} + +resource frontend 'Applications.Core/containers@2023-10-01-preview' = { + name: 'frontend' + properties: { + application: app.id + container: { + image: 'myapp/frontend:v1.0.0' + ports: { + web: { containerPort: 3000 } + } + } + connections: { + backend: { source: backend.id } + } + } +} + +resource backend 'Applications.Core/containers@2023-10-01-preview' = { + name: 'backend' + properties: { + application: app.id + container: { + image: 'myapp/backend:v1.0.0' + ports: { + api: { containerPort: 8080 } + } + } + connections: { + cache: { source: cache.id } + } + } +} + +resource cache 'Applications.Datastores/redisCaches@2023-10-01-preview' = { + name: 'cache' + properties: { + application: app.id + environment: environment + } +} +``` + +Generate the app graph: + +```bash +rad app graph app.bicep +``` + +This creates `.radius/app-graph.json` with the full topology. + +## 2. View the Output + +The JSON output shows resources and their connections: + +```json +{ + "metadata": { + "generatedAt": "2026-02-04T00:58:00Z", + "radiusCliVersion": "0.35.0", + "sourceFiles": ["app.bicep"], + "sourceHash": "sha256:abc123...", + "gitCommit": "def456" + }, + "resources": [ + { + "id": ".../Applications.Core/containers/frontend", + "name": "frontend", + "type": "Applications.Core/containers", + "gitInfo": { + "commitSha": "def456...", + "author": "you@example.com", + "date": "2026-02-03T15:30:00Z", + "message": "Add frontend container" + } + } + // ... more resources + ], + "connections": [ + { + "sourceId": ".../containers/frontend", + "targetId": ".../containers/backend", + "type": "connection" + } + // ... more connections + ] +} +``` + +## 3. Generate Markdown Preview + +For a human-readable preview with Mermaid diagram: + +```bash +rad app graph app.bicep --format markdown +``` + +This creates both `.radius/app-graph.json` and `.radius/app-graph.md`: + +```markdown +# App Graph: myapp + +## Resources + +| Name | Type | Source | Last Commit | +|------|------|--------|-------------| +| frontend | Applications.Core/containers | app.bicep:12 | [def456](../../commit/def456) | +| backend | Applications.Core/containers | app.bicep:25 | [def456](../../commit/def456) | +| cache | Applications.Datastores/redisCaches | app.bicep:40 | [abc123](../../commit/abc123) | + +## Topology + +\```mermaid +graph LR + frontend[frontend] + backend[backend] + cache[(cache)] + + frontend --> backend + backend --> cache +\``` +``` + +## 4. Commit the Graph + +The app graph is designed to be committed alongside your Bicep files: + +```bash +git add app.bicep .radius/app-graph.json +git commit -m "Add Redis cache to application" +``` + +## 5. Set Up GitHub Action for PR Diffs + +Add `.github/workflows/app-graph-diff.yml`: + +```yaml +name: App Graph Diff + +on: + pull_request: + paths: + - '**/.radius/app-graph.json' + push: + branches: + - main + paths: + - '**/.radius/app-graph.json' + +permissions: + pull-requests: write + +jobs: + diff: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - uses: radius-project/app-graph-diff-action@v1 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} +``` + +When a PR changes the app graph, the Action posts a comment showing: +- Added/removed/modified resources +- New/removed connections +- Before/after Mermaid diagrams + +## 6. Validate Graph Freshness in CI + +Add validation to catch stale graphs: + +```yaml +name: Validate App Graph + +on: + pull_request: + paths: + - '**/*.bicep' + - '**/.radius/app-graph.json' + +jobs: + validate: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Install Radius CLI + run: | + curl -fsSL https://get.radapp.io/install.sh | bash + rad bicep download + + - name: Validate graph is current + run: | + rad app graph app.bicep --stdout > /tmp/expected.json + diff .radius/app-graph.json /tmp/expected.json || { + echo "::error::App graph is stale. Run 'rad app graph app.bicep' and commit." + exit 1 + } +``` + +## Common Options + +| Option | Description | +|--------|-------------| +| `--stdout` | Write JSON to stdout instead of file | +| `-o ` | Write to custom output path | +| `--format markdown` | Also generate Markdown preview | +| `--no-git` | Skip git metadata (faster) | +| `--parameters ` | Use parameter file for Bicep | +| `--at ` | Generate graph at specific commit | + +## Troubleshooting + +### "Bicep CLI not found" + +```bash +rad bicep download +``` + +### "Not a git repository" + +Git metadata is optional. The graph generates successfully, but `gitInfo` fields show "not available". + +### "Stale graph detected in CI" + +Regenerate and commit: + +```bash +rad app graph app.bicep +git add .radius/app-graph.json +git commit --amend --no-edit +git push --force-with-lease +``` + +## Next Steps + +- [View graph history](./history.md) - Track architecture evolution over time +- [Compare environments](./environments.md) - See how portable types resolve differently +- [Customize the GitHub Action](./github-action.md) - Advanced configuration options diff --git a/specs/003-git-app-graph-storage/radius-graph-proposal.md b/specs/003-git-app-graph-storage/radius-graph-proposal.md new file mode 100644 index 0000000000..e83438883f --- /dev/null +++ b/specs/003-git-app-graph-storage/radius-graph-proposal.md @@ -0,0 +1,98 @@ +# Deployment Lineage: Radius as a Versioned Infrastructure Graph + +## Summary + +Given any running resource deployed by Radius, a user can answer: *What exact versions of which files, recipes, and configurations produced this?* Given any source artifact, a user can answer: *What did this deploy, and where?* This proposal introduces a versioned graph that makes this possible — connecting every deployed resource to the specific versions of every artifact that created it: the `app.bicep` file, the parameters, the recipes, the environment configuration, and the deployment operation itself. + +## Context + +Radius is currently developing three capabilities that shape this proposal: + +1. **App graph visualization** — displaying `app.bicep` files as interactive graphs. +2. **Deployment graph display** — showing the live state of a deployed application as a graph. +3. **Repo Radius** — running Radius inside GitHub Actions without a persistent control plane, using serialized files stored in orphan git branches. + +These features will work using the existing storage backends (etcd, serialized JSON files, or PostgreSQL). This proposal is to develop, in parallel, a graph storage system that unifies these into a single connected model where every deployed resource is linked to the versioned artifacts that produced it. + +## Proposal: A Versioned Graph of Deployed State and Source Artifacts + +Given any deployed resource, a user can traverse the graph to find: + +- The exact `app.bicep` file (and version) that declared it +- The artifact diffs between deployments +- The parameter values used at deployment time +- The recipe and recipe pack that provisioned the underlying infrastructure +- The environment and group configuration in effect when it was deployed +- The deployment operation that created or last modified it + +Given any source artifact (an `app.bicep` file, a recipe, a parameter set), a user can traverse the graph in the other direction to find every deployed resource it has ever produced, across environments and over time. + +```mermaid +flowchart TB + subgraph Source["Source Artifacts (versioned)"] + Bicep["app.bicep
(v3)"] + Params["Parameters
(v2)"] + Recipe["Recipe Pack
(v1.4)"] + EnvConfig["Environment
Config (v5)"] + end + + Deploy["Deployment
Operation #42"] + + subgraph Live["Deployed State"] + Container["frontend
container"] + Redis["redis
cache"] + Gateway["gateway
router"] + end + + Bicep --> Deploy + Params --> Deploy + Recipe --> Deploy + EnvConfig --> Deploy + Deploy --> Container + Deploy --> Redis + Deploy --> Gateway +``` + +*Starting from any node, you can navigate to every other node. What created this container? What did this recipe deploy? What changed between deployment #41 and #42?* + +## Current vs. Future Storage + +### Current storage + +```mermaid +flowchart LR + State["Radius State"] --> db["etcd / PostgreSQL"] + AppViz["App.bicep Visualization"] --> appjson["App JSON"] + DeployViz["Deployed Graph"] --> deployjson["Deployment JSON"] +``` + +Deployed state, source artifacts, and visualization data live in separate stores with no connecting relationships. + +### Future graph storage + +```mermaid +flowchart LR + State["Radius State"] --> Graph["Versioned Graph"] + AppBicep["App.bicep"] --> Graph + Deployed["Deployed State"] --> Graph + SupplyChain["Supply Chain"] --> Graph + GitOpsConfig["GitOps Configuration"] --> Graph +``` + +One versioned graph where deployed state, source artifacts, and deployment history are connected and traversable. + +## High-Level Features + +- **Versioned graph state** Every deployment produces a versioned snapshot of the graph — source artifacts, deployment operations, and resulting deployed resources — linked together. Users can browse, query, and diff the graph across any point in its history. + +- **Graph querying** The storage layer supports graph traversal queries for navigating the relationships between deployed state and source artifacts. + +- **Multiple runtime modes** The graph storage system runs as: + + | Mode | Description | Example | + | ------ | ------------- | --------- | + | **Local stateless** | Graph generated from files in a local repository clone | Developer preview, pre-commit validation | + | **Server stateless** | Graph generated ephemerally in CI/CD without a persistent process | GitHub Actions, PR visualization | + | **Long-running stateful** | Graph hosted in a graph database on the control plane | Enterprise deployments, real-time collaboration | + +- **Ingest from existing tools** Many infrastructure and application tools already store their configuration in text files (YAML, JSON, HCL, Bicep) and ship parsers that turn those files into structured data. GitHub's dependency graph works this way — it reads package manifests from many platforms and builds a unified dependency view. Radius can use the same approach: leverage existing parsers from tools like Kustomize, KRO, Flux, and others to pull their resources into the graph alongside Radius-native `app.bicep` definitions. No new file formats are required — the graph simply understands what's already in a repository. diff --git a/specs/003-git-app-graph-storage/research.md b/specs/003-git-app-graph-storage/research.md new file mode 100644 index 0000000000..b25cdc6087 --- /dev/null +++ b/specs/003-git-app-graph-storage/research.md @@ -0,0 +1,239 @@ +# Research: Git App Graph Preview + +**Feature Branch**: `001-git-app-graph-preview` +**Date**: February 4, 2026 + +## Research Tasks + +### 1. Radius CLI Architecture + +**Decision**: Extend existing `rad app graph` command in `pkg/cli/cmd/app/graph/` + +**Rationale**: +- The existing `rad app graph ` command already follows established CLI patterns +- Command structure uses Cobra framework with Runner pattern (`NewCommand()` + `Runner.Run()`) +- Output formatting is handled through `pkg/cli/output/` utilities +- Adding file-based input preserves conceptual consistency ("both are app graphs") + +**Alternatives Considered**: +- New `rad graph` top-level command: Rejected (breaks CLI hierarchy, less discoverable) +- New `rad bicep graph` command: Rejected (conceptually these are both "app graphs", just from different sources) + +### 2. Existing App Graph Data Structures + +**Decision**: Extend `ApplicationGraphResponse` and related types in `pkg/corerp/api/` + +**Rationale**: +- Existing types (`ApplicationGraphResource`, `ApplicationGraphConnection`) capture most needed fields +- Adding git metadata fields is additive and backward-compatible +- Deterministic output requires adding `sourceHash`, `sourceFile`, `sourceLine` metadata +- Types are already JSON-serializable via auto-generated marshalling code + +**Existing Structure** (from `zz_generated_models.go`): +```go +type ApplicationGraphResponse struct { + Resources []*ApplicationGraphResource +} + +type ApplicationGraphResource struct { + ID *string + Name *string + Type *string + ProvisioningState *string + Connections []*ApplicationGraphConnection + OutputResources []*ApplicationGraphOutputResource +} +``` + +**Additions Needed**: +- `GitInfo` struct: commit SHA, author, date, message +- `SourceFile`, `SourceLine` for Bicep source tracking +- `Metadata` struct: generatedAt, sourceFiles, sourceHash, radiusCliVersion + +### 3. Bicep Parsing Approach + +**Decision**: Use official Bicep CLI for compilation, then parse ARM JSON + +**Rationale** (per Constitution Principle VII - Simplicity Over Cleverness): +- Bicep CLI provides `bicep build --stdout` to compile to ARM JSON +- Radius already has Bicep CLI integration in `pkg/cli/bicep/` +- ARM JSON is stable and well-documented; parsing it avoids Bicep grammar complexity +- External modules are resolved by Bicep CLI, not our code + +**Alternatives Considered**: +- Custom Bicep parser: Rejected (complex, maintenance burden, grammar changes) +- ANTLR-based parser: Rejected (over-engineering for this use case) +- Use Bicep language server: Rejected (heavyweight, overkill for static analysis) + +**Implementation Pattern**: +```go +// Existing pattern in pkg/cli/bicep/types.go +func (impl *Impl) PrepareTemplate(filePath string) (map[string]any, error) { + args := []string{"build", "--stdout", filePath} + // Execute bicep CLI and parse JSON output +} +``` + +### 4. Graph Extraction from ARM JSON + +**Decision**: Extract resources and connections from compiled ARM JSON template + +**Rationale**: +- ARM JSON has stable schema with `resources` array +- Radius resources use `connections` and `routes` properties for relationships +- Existing graph logic in `pkg/corerp/frontend/controller/applications/graph_util.go` shows patterns + +**Key Extraction Points**: +1. `resources[].type` - Resource type identification +2. `resources[].name` - Resource name (may contain expressions) +3. `resources[].properties.connections` - Direct connections +4. `resources[].properties.routes` - Gateway routes +5. `resources[].dependsOn` - Explicit dependencies + +### 5. Git Integration + +**Decision**: Use `git log` and `git blame` via exec, not library + +**Rationale**: +- Git is universally available in development environments +- Shell commands are simpler than CGo bindings to libgit2 +- Radius already uses exec patterns for Bicep CLI +- Graceful degradation when not in git repo or shallow clone + +**Commands Needed**: +- `git blame -l -e ` - Get commit SHA per line +- `git log -1 --format='%H|%ae|%aI|%s' ` - Get commit metadata +- `git rev-parse --show-toplevel` - Detect git repo root + +### 6. GitHub Action Architecture + +**Decision**: Lightweight Action that reads committed JSON from git history; no graph generation + +**Rationale** (per spec Committed Artifact Model): +- Action only needs git and jq, not Bicep/Radius tooling +- Works in forks without special secrets +- Fast execution (JSON comparison vs. full compilation) +- Reproducible (graph captured at commit time) + +**Implementation**: +1. Checkout base and head commits +2. Read `.radius/app-graph.json` from each +3. Compute diff using JSON comparison +4. Render diff as Markdown with Mermaid diagrams +5. Post/update PR comment using `peter-evans/create-or-update-comment` + +**Alternatives Considered**: +- GitHub App: Rejected for MVP (more complex setup, centralized management) +- Generate graph in CI: Rejected (requires Bicep tooling, slower, less reproducible) + +### 7. Diff Computation Strategy + +**Decision**: JSON-based diffing with semantic resource comparison + +**Rationale**: +- JSON is deterministic when keys are sorted +- Resource ID provides stable identity across commits +- Diff should show added/removed/modified at resource level, not line level + +**Diff Algorithm**: +1. Parse base and head JSON +2. Create resource map keyed by ID +3. Compare: + - Added: ID in head but not base + - Removed: ID in base but not head + - Modified: ID in both but properties differ +4. For connections: compare by (sourceID, targetID) tuples + +### 8. Output Formats + +**Decision**: JSON canonical, Markdown additive with embedded Mermaid + +**Rationale** (per spec): +- JSON is machine-readable, deterministic, diffable +- Markdown renders in GitHub UI without additional tooling +- Mermaid diagrams supported natively by GitHub +- Separation allows different consumers (CI vs. humans) + +**JSON Schema**: +```json +{ + "metadata": { + "generatedAt": "2026-01-30T10:15:00Z", + "sourceFiles": ["app.bicep", "modules/database.bicep"], + "sourceHash": "sha256:abc123...", + "radiusCliVersion": "0.35.0" + }, + "resources": [...], + "connections": [...] +} +``` + +**Mermaid Shapes** (per spec): +- Containers: rectangles (`[name]`) +- Gateways: diamonds (`{name}`) +- Databases: cylinders (`[(name)]`) + +### 9. Platform Abstraction for Future Integrations + +**Decision**: Separate diff computation from rendering; platform-specific rendering layer + +**Rationale** (per user input: "design choices today won't limit us in the future"): +- Core diff logic (JSON comparison) is platform-agnostic +- GitHub-specific code isolated to: + - GitHub Action (workflow YAML) + - Markdown/Mermaid rendering + - PR comment posting +- Future GitLab integration would only need new rendering layer + +**Architecture**: +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Platform-Agnostic │ +├─────────────────────────────────────────────────────────────────┤ +│ CLI (rad app graph) │ JSON Schema │ Diff Computation │ +│ Git Integration │ Data Model │ Core Rendering (MD) │ +└─────────────────────────────────────────────────────────────────┘ + │ + ┌─────────┴─────────┐ + ▼ ▼ + ┌───────────────┐ ┌───────────────┐ + │ GitHub Action │ │ GitLab CI │ + │ PR Comments │ │ MR Notes │ + │ Mermaid │ │ (Future) │ + └───────────────┘ └───────────────┘ +``` + +### 10. Radius Bicep Extension Compatibility + +**Decision**: Support Radius Bicep extension type definitions + +**Rationale**: +- Radius extends Bicep with custom types (`Applications.Core/containers`, etc.) +- These types must be recognized in ARM JSON output +- Type registry already exists in `pkg/corerp/api/` + +**Implementation**: +- Resource type detection checks for `Applications.*` prefix +- Portable types (`Radius.Data/store`) shown as-is in static graph +- Environment-resolved graph (P3) requires live Radius environment connection + +## Technology Decisions Summary + +| Area | Decision | Rationale | +|------|----------|-----------| +| CLI Framework | Cobra (existing) | Consistency with `rad` CLI | +| Bicep Parsing | Bicep CLI → ARM JSON | Simplicity, official support | +| Git Operations | Shell exec | Universal, no CGo | +| Data Structures | Extend existing types | Backward compatible | +| GitHub Integration | Action, not App | Simpler setup, fork-friendly | +| Diff Algorithm | JSON semantic diff | Deterministic, meaningful | +| Output | JSON + optional Markdown | Machine + human readable | +| Platform Abstraction | Rendering layer separation | Future GitLab support | + +## Open Questions Resolution + +1. **Bicep Compiler Integration**: ✅ Use official Bicep CLI (per Constitution Principle VII) +2. **Graph Storage**: ✅ Committed to `.radius/app-graph.json` (per spec) +3. **GitHub App vs Action**: ✅ GitHub Action for MVP (simpler, fork-friendly) +4. **Diff Visualization**: Table + Mermaid in PR comments +5. **Parameter Handling**: ✅ Require params file; fail with clear error listing missing required parameters if Bicep has required parameters (no defaults) but no `--parameters` flag provided diff --git a/specs/003-git-app-graph-storage/spec.md b/specs/003-git-app-graph-storage/spec.md new file mode 100644 index 0000000000..28197dd929 --- /dev/null +++ b/specs/003-git-app-graph-storage/spec.md @@ -0,0 +1,430 @@ +# Feature Specification: Git App Graph Preview + +**Feature Branch**: `001-git-app-graph-preview` +**Created**: January 30, 2026 +**Status**: Draft +**Input**: User description: "Radius currently stores the state of application deployments as an app graph within its data store. Today, the app graph does not get generated until the application is deployed. Help me build an app graph representation for applications that are defined (e.g. in an app.bicep file) but not yet deployed. Additionally, enrich the app graph representation with git changelog info (i.e. git commit data) so that I may use this data to visualize how the app graph changes over time (i.e. across commits). The ultimate goal is to be able to visualize the app graph and do diffs of the app graph in GitHub on PRs, commit comparisons, etc." + +## Clarifications + +### Session 2026-02-04 + +- Q: GitHub App vs Action for PR integration? → A: GitHub Action (fork-friendly, no installation approval required, aligns with existing Radius workflow patterns) +- Q: Diff visualization format in PR comments? → A: Table + Mermaid diagrams (change table for details, before/after diagrams for visual topology) +- Q: How to handle Bicep parameters without params file? → A: Require params file (fail with error if Bicep has required parameters but no `--parameters` provided) +- Q: GitHub Action trigger events? → A: `pull_request` + `push` (PR for review comments, push to main for baseline tracking) +- Q: Monorepo support with multiple app graphs? → A: Auto-detect all `**/.radius/app-graph.json` files; each diffed independently + +## Problem Statement + +Radius currently generates application graphs only after deployment, which means: +1. Developers cannot preview the app graph structure before deploying +2. There's no way to track how the application architecture evolves over time +3. PR reviewers cannot see the impact of Bicep changes on the overall application topology +4. No mechanism exists to compare app graphs across commits or branches + +This feature introduces **static app graph generation** from Bicep files and **git-aware graph versioning** to enable visualization and diffing in GitHub workflows. + +## User Scenarios & Testing *(mandatory)* + +### User Story 1 - Generate App Graph from Bicep Files (Priority: P1) + +As a **developer**, I want to generate an app graph from my `app.bicep` file without deploying, so I can preview the application topology and validate my changes locally. + +As a **platform engineer**, I want to review app graph changes in PRs, so I can ensure architectural changes align with organizational standards before deployment. + +**Why this priority**: This is the foundational capability. Without static graph generation, no other features can function. It delivers immediate value by enabling local validation. + +**Independent Test**: Can be fully tested by running a CLI command against a Bicep file and verifying the graph output matches expected structure. + +**Acceptance Scenarios**: + +1. **Given** a valid `app.bicep` file with container, gateway, and database resources, **When** I run `rad app graph app.bicep`, **Then** I receive a JSON graph representation showing all resources and their connections. + +2. **Given** a Bicep file with syntax errors, **When** I run `rad app graph app.bicep`, **Then** I receive a clear error message indicating the parsing failure with line/column information. + +3. **Given** a Bicep file referencing external modules, **When** I run `rad app graph app.bicep`, **Then** the graph includes resources from all referenced modules with proper dependency tracking. + +4. **Given** a Bicep file with parameterized values, **When** I run `rad app graph app.bicep --parameters params.json`, **Then** the graph reflects the resolved parameter values. + +5. **Given** a Bicep file with required parameters (no defaults) and no `--parameters` flag, **When** I run `rad app graph app.bicep`, **Then** I receive a clear error listing the missing required parameters. + +6. **Given** a Bicep file using the Radius Bicep extension types, **When** I run `rad app graph app.bicep`, **Then** the graph correctly identifies Radius-specific resource types and their relationships. + +--- + +### User Story 2 - Export Graph as Diff-Friendly Format (Priority: P1) + +As a developer, I want the app graph exported in a deterministic, diff-friendly format, so I can commit it to version control and see meaningful diffs when it changes. + +**Why this priority**: Critical for enabling GitHub integration. Without a stable, diffable format, PR visualization is impossible. + +**Independent Test**: Generate graph twice from identical Bicep, verify outputs are byte-identical. Modify Bicep, regenerate, verify diff shows only the changed elements. + +**Output Model**: JSON is the canonical data format, always generated. Markdown is a rendered preview of the JSON data, generated additively when requested. + +**Acceptance Scenarios**: + +1. **Given** an app graph, **When** I export it, **Then** the JSON output is deterministically sorted (alphabetical by resource ID) producing identical output for identical inputs. + +2. **Given** an app graph, **When** I run `rad app graph app.bicep`, **Then** JSON is written to `.radius/app-graph.json` (default location) serving as the single source of truth for all automation and diff operations. + +3. **Given** an app graph, **When** I run `rad app graph app.bicep --stdout`, **Then** JSON is written to stdout instead of a file. + +4. **Given** an app graph, **When** I run `rad app graph app.bicep --format markdown`, **Then** I receive **both** `.radius/app-graph.json` and `.radius/app-graph.md` containing: + - A resource table with name, type, source file, and git metadata + - An embedded Mermaid diagram showing the topology that GitHub renders automatically + +5. **Given** a graph exported to markdown, **When** viewed in GitHub, **Then** the Mermaid diagram renders as a visual flowchart with distinct shapes for resource types (containers as rectangles, gateways as diamonds, databases as cylinders). + +6. **Given** two app graphs from different commits, **When** I diff them, **Then** the diff is computed from JSON (not Markdown), and added/removed/modified resources are clearly identified. + +--- + +### User Story 3 - Git Metadata Enrichment (Priority: P2) + +As a developer, I want the app graph to automatically include git commit information, so I can track when and why each resource was added or modified. + +**Why this priority**: Builds on P1 capabilities to enable historical tracking. Valuable but not blocking core functionality. + +**Independent Test**: Generate graph from a Bicep file in a git repository, verify each resource includes commit SHA, author, and timestamp of last modification by default. + +**Acceptance Scenarios**: + +1. **Given** a Bicep file in a git repository, **When** I run `rad app graph app.bicep`, **Then** each resource automatically includes the commit SHA, author, date, and message of its last modification. + +2. **Given** a resource defined across multiple Bicep files, **When** I generate the graph, **Then** the resource shows the most recent commit that affected any of its defining files. + +3. **Given** a newly added resource not yet committed, **When** I generate the graph, **Then** the resource is marked as "uncommitted" with the current working directory state. + +4. **Given** a graph with git metadata, **When** I export to markdown, **Then** each resource row includes a linked commit SHA (e.g., `[abc123](../../commit/abc123)`). + +5. **Given** a Bicep file in a git repository, **When** I run `rad app graph app.bicep --no-git`, **Then** the graph is generated without git metadata for faster execution. + +6. **Given** a Bicep file outside a git repository, **When** I run `rad app graph app.bicep`, **Then** the graph is generated successfully with git fields marked as "not available". + +--- + +### User Story 4 - GitHub Action for PR Graph Diff (Priority: P2) + +As a PR reviewer, I want to see a visual diff of the app graph in PR comments, so I can understand the architectural impact of code changes without deploying. + +**Why this priority**: High-value GitHub integration, but depends on P1 capabilities being stable. + +**Operational Model**: The GitHub Action reads committed `.radius/app-graph.json` files from git history — it does NOT generate graphs on-demand. This keeps the Action lightweight (no Bicep/Radius tooling required) and fast. + +**Trigger Events**: The Action supports two trigger modes: +- **`pull_request`**: Posts diff comments on PRs when `.radius/app-graph.json` changes +- **`push` to main/default branch**: Updates baseline tracking for historical comparison + +**Monorepo Support**: The Action auto-detects all `**/.radius/app-graph.json` files in the repository. Each graph is diffed independently, with separate PR comment sections per application. + +**Independent Test**: Create a PR with Bicep changes and updated graph JSON, verify the action posts a comment showing before/after graph comparison. + +**Acceptance Scenarios**: + +1. **Given** a PR that includes changes to `.radius/app-graph.json`, **When** the GitHub Action runs, **Then** it reads the JSON from base and head commits and posts a comment showing the graph diff with added/removed/modified resources highlighted. + +2. **Given** a PR with no changes to `.radius/app-graph.json`, **When** the GitHub Action runs, **Then** it posts a comment indicating "No app graph changes detected." + +3. **Given** a PR that adds a new connection between resources, **When** the GitHub Action runs, **Then** the diff clearly shows the new connection with source and target resources. + +4. **Given** a PR comment already exists from a previous run, **When** the PR is updated and the action runs again, **Then** the existing comment is updated rather than creating a duplicate. + +5. **Given** a PR where Bicep files changed but `.radius/app-graph.json` was not updated, **When** the CI validation job runs, **Then** it fails with a message instructing the developer to run `rad app graph app.bicep` and commit the updated graph. + +6. **Given** a monorepo with multiple Radius applications (e.g., `apps/frontend/.radius/app-graph.json` and `apps/backend/.radius/app-graph.json`), **When** the GitHub Action runs on a PR, **Then** it detects all graph files and posts a unified comment with separate diff sections per application. + +--- + +### User Story 5 - Historical Graph Timeline (Priority: P3) + +As a developer, I want to view how my app graph evolved across commits, so I can understand architectural decisions and identify when changes were introduced. + +**Why this priority**: Advanced feature for historical analysis. Valuable for debugging and auditing but not essential for core workflow. + +**Independent Test**: Generate timeline for last 10 commits, verify each entry shows the graph state and changes from previous commit. + +**Acceptance Scenarios**: + +1. **Given** a git repository with multiple commits affecting Bicep files, **When** I run `rad app graph history app.bicep --commits 10`, **Then** I receive a timeline showing graph snapshots at each commit with change summaries. + +2. **Given** a specific commit SHA, **When** I run `rad app graph app.bicep --at abc123`, **Then** I receive the app graph as it existed at that commit. + +3. **Given** two commit SHAs, **When** I run `rad app graph diff app.bicep --from abc123 --to def456`, **Then** I receive a detailed diff showing all graph changes between those commits. + +--- + +### User Story 6 - Environment-Resolved Graph (Priority: P3) + +As a platform engineer, I want to see how abstract Radius types resolve to concrete infrastructure in a specific environment, so I can understand the actual resources that will be deployed. + +**Why this priority**: Advanced feature for environment-specific analysis. The static graph (showing portable types) serves most PR review needs; resolved views are valuable for deployment planning and troubleshooting. + +**Background**: Radius portable types like `Radius.Data/store` resolve differently depending on the environment's recipe configuration: +- Environment → RecipePack → Recipe → Concrete Resource +- The same `Radius.Data/store` might become PostgreSQL in `dev` and CosmosDB in `prod` + +**Independent Test**: Generate resolved graph for an environment with known recipe bindings, verify concrete resource types appear instead of abstract Radius types. + +**Acceptance Scenarios**: + +1. **Given** a Bicep file with `Radius.Data/store` and a connected Radius environment with PostgreSQL recipes, **When** I run `rad app graph app.bicep --environment prod`, **Then** the graph shows the resolved `Azure.DBforPostgreSQL/flexibleServers` (or equivalent) instead of the abstract `Radius.Data/store`. + +2. **Given** a Bicep file with portable types, **When** I run `rad app graph app.bicep --environment dev` and `rad app graph app.bicep --environment prod`, **Then** I can compare how the same application resolves to different infrastructure across environments. + +3. **Given** an environment where a recipe is not configured for a portable type, **When** I run `rad app graph app.bicep --environment prod`, **Then** the graph shows the abstract type with an annotation indicating "no recipe bound". + +4. **Given** a Bicep file, **When** I run `rad app graph app.bicep` (no `--environment` flag), **Then** the graph shows the abstract portable types (default behavior unchanged). + +--- + +### Edge Cases + +- What happens when Bicep file references resources outside the current file/module that cannot be resolved? + - Generate partial graph with unresolved references marked as "external" placeholders +- How does the system handle circular dependencies in Bicep? + - Detect and report cycles with clear error messaging; still generate graph with cycle annotation +- What happens when git history is shallow (e.g., `--depth 1` clone)? + - Gracefully degrade: use available history, mark resources as "history unavailable" when git blame fails +- How does the system handle large graphs (100+ resources)? + - Paginate CLI output, provide `--filter` options, optimize JSON/Markdown output for size +- What happens when Bicep uses runtime expressions that can't be statically resolved? + - Mark affected values as "dynamic" in the graph, use placeholder notation +- What happens when Bicep files use cloud-specific resources (Azure, AWS)? + - Graph generation MUST work regardless of cloud provider; cloud-specific resources are represented with their provider prefix (e.g., `Microsoft.Storage/storageAccounts`, `AWS::S3::Bucket`) +- What happens when the committed graph is stale (Bicep changed but graph not regenerated)? + - CI validation job compares committed graph to freshly generated graph; fails PR if they differ + - Graph JSON includes `sourceHash` to detect staleness without full regeneration + - Clear error message instructs developer to run `rad app graph app.bicep` +- What does the graph show for portable Radius types like `Radius.Data/store` that resolve differently per environment? + - **Static graph shows abstract types**: The declared `Radius.Data/store` is shown, not the resolved infrastructure (PostgreSQL, CosmosDB, etc.) + - This is intentional—the static graph represents the **portable application architecture** independent of environment-specific recipe resolution + - For environment-resolved views, see User Story 6 (P3) + +--- + +## CLI Design + +This feature extends the existing `rad app graph` command with file-based input for static graph generation. The command intelligently distinguishes between deployed apps and Bicep files based on the argument: + +| Command | Input Type | Output | +|---------|------------|--------| +| `rad app graph myapp` | App name | Deployed app graph (existing behavior) | +| `rad app graph myapp -e prod` | App name + environment | Deployed graph in specific environment | +| `rad app graph app.bicep` | Bicep file (`.bicep` extension) | JSON to `.radius/app-graph.json` (default) | +| `rad app graph app.bicep --stdout` | Bicep file + stdout flag | JSON to stdout (no file written) | +| `rad app graph app.bicep -o custom.json` | Bicep file + custom output | JSON to specified file | +| `rad app graph app.bicep --format markdown` | Bicep file + markdown | JSON + Markdown to `.radius/` | +| `rad app graph app.bicep --no-git` | Bicep file + no-git | JSON without git metadata (faster) | +| `rad app graph app.bicep --at abc123` | Bicep file + commit | JSON at specific commit | +| `rad app graph diff app.bicep --from abc123 --to def456` | Bicep file + commits | Diff computed from JSON, output as JSON or Markdown | +| `rad app graph history app.bicep --commits 10` | Bicep file + count | Historical timeline | +| `rad app graph app.bicep --environment prod` | Bicep file + environment | JSON with resolved recipe types | + +**Output Model**: +- **JSON is canonical**: Always generated, serves as the single source of truth for all automation and diff operations +- **Markdown is additive**: When `--format markdown` is specified, Markdown is generated *in addition to* JSON as a human-readable preview +- **GitHub Action uses JSON**: Diff computation is always JSON-to-JSON; Markdown is purely a rendering/presentation layer for PR comments + +**Design Rationale**: Unifying under `rad app graph` provides: +- Conceptual consistency: both are "app graphs" (prospective vs. deployed) +- Discoverability: all graph functionality in one place +- Intuitive disambiguation: `.bicep` extension clearly indicates file input +- Alignment with existing `rad app graph ` pattern + +--- + +## Committed Artifact Model + +The app graph JSON is designed to be **committed to version control** as a tracked artifact. This enables lightweight GitHub integration without requiring the Action to have Bicep/Radius tooling. + +### Default Output Location + +By default, `rad app graph app.bicep` writes to `.radius/app-graph.json` relative to the Bicep file's directory: + +``` +myapp/ +├── app.bicep +├── modules/ +│ └── database.bicep +└── .radius/ + ├── app-graph.json # Canonical graph data (committed) + └── app-graph.md # Optional preview (if --format markdown) +``` + +### Developer Workflow + +```bash +# 1. Make changes to Bicep files +vim app.bicep + +# 2. Regenerate the graph (writes to .radius/app-graph.json by default) +rad app graph app.bicep + +# 3. Commit both the Bicep changes and updated graph +git add app.bicep .radius/app-graph.json +git commit -m "Add redis cache to application" + +# 4. Push and create PR — GitHub Action reads committed JSON to render diff +git push +``` + +### Why Committed Artifacts? + +| Benefit | Explanation | +|---------|-------------| +| **Simple GitHub Action** | Action is a lightweight viewer that reads JSON from git history — no Bicep CLI, no Radius environment needed | +| **Fast CI** | No graph generation in CI; diff is just JSON comparison | +| **Reproducible** | Graph captured at commit time, not regenerated with potentially different tooling | +| **Auditable** | Graph evolution visible in git history alongside code changes | +| **Fork-friendly** | Works in forks without special tooling or secrets | + +### Staleness Detection + +To prevent committed graphs from drifting out of sync with Bicep files: + +1. **CI Validation Job** (recommended): Regenerate graph in CI, compare to committed version, fail if different +2. **Pre-commit Hook** (optional): Validate graph matches Bicep before allowing commit +3. **Graph Metadata**: JSON includes `sourceHash` field — hash of input Bicep file(s) for staleness detection + +```json +{ + "metadata": { + "generatedAt": "2026-01-30T10:15:00Z", + "sourceFiles": ["app.bicep", "modules/database.bicep"], + "sourceHash": "sha256:abc123...", + "radiusCliVersion": "0.35.0" + }, + "resources": [...], + "connections": [...] +} +``` + +--- + +## Requirements *(mandatory)* + +### Functional Requirements + +- **FR-001**: System MUST parse Bicep files and extract resource definitions without requiring deployment +- **FR-002**: System MUST resolve module references and build a complete graph across multiple Bicep files +- **FR-003**: System MUST extract connection relationships from resource properties (connections, routes, ports) +- **FR-004**: System MUST produce deterministic output (same input = byte-identical output) +- **FR-005**: System MUST always generate JSON output as the canonical data format with stable key ordering for deterministic diffs +- **FR-006**: System MUST support Markdown output as an **additive** preview format (generated alongside JSON when `--format markdown` is specified), containing a resource table and embedded Mermaid diagram +- **FR-007**: System MUST perform all diff computations using JSON data, with Markdown used only as a rendering layer for human consumption +- **FR-008**: System MUST enrich graph nodes with git metadata (commit SHA, author, date, message) by default when in a git repository, with `--no-git` flag to disable +- **FR-009**: System MUST track which Bicep file(s) define each resource for git blame integration +- **FR-010**: System MUST write graph output to `.radius/app-graph.json` by default (relative to Bicep file location), with `--stdout` flag for stdout output and `-o` flag for custom path +- **FR-011**: System MUST include `sourceHash` metadata in JSON output to enable staleness detection +- **FR-012**: System MUST provide a GitHub Action that reads committed graph JSON from git history and posts graph diffs on PRs (no graph generation in CI) +- **FR-013**: System MUST update existing PR comments rather than creating duplicates +- **FR-014**: System MUST support generating graphs at specific git commits/refs +- **FR-015**: System MUST handle Bicep parameter files to resolve parameterized values +- **FR-016**: System MUST report clear errors for invalid Bicep syntax with file/line/column information +- **FR-017**: System MUST handle unresolvable references gracefully with placeholder annotations +- **FR-018**: System MUST work with Bicep files targeting any cloud provider (multi-cloud neutrality per Constitution Principle III) +- **FR-019**: System MUST be compatible with the Radius Bicep extension type definitions + +### Non-Functional Requirements + +- **NFR-001**: All Go code MUST follow Effective Go patterns and pass `golangci-lint` (Constitution Principle II) +- **NFR-002**: All exported Go packages, types, and functions MUST have godoc comments (Constitution Code Quality Standards) +- **NFR-003**: Feature MUST NOT require changes to existing deployment workflows (Constitution Principle IX - Incremental Adoption) +- **NFR-004**: CLI commands MUST follow existing `rad` CLI patterns and conventions +- **NFR-005**: Error messages MUST be actionable with clear guidance for resolution (Constitution Principle VI) + +### Key Entities + +- **AppGraph**: Root container holding all resources, connections, and metadata for a single application + - Resources: Collection of AppGraphResource nodes + - Metadata: Git commit info, generation timestamp, source files + +- **AppGraphResource**: Single resource node in the graph + - ID: Unique resource identifier (matches Radius resource ID format) + - Name: Human-readable resource name + - Type: Resource type (e.g., `Applications.Core/containers`) + - SourceFile: Bicep file path where resource is defined + - SourceLine: Line number in source file + - Connections: Outbound connections to other resources + - GitInfo: Last commit SHA, author, date, message affecting this resource + +- **AppGraphConnection**: Edge between two resources + - SourceID: Origin resource + - TargetID: Destination resource + - Direction: Outbound/Inbound + - Type: Connection type (connection, route, port binding) + +- **GraphDiff**: Comparison result between two graphs + - AddedResources: Resources present in new but not old + - RemovedResources: Resources present in old but not new + - ModifiedResources: Resources with changed properties/connections + - AddedConnections: New edges + - RemovedConnections: Removed edges + +--- + +## Success Criteria *(mandatory)* + +### Measurable Outcomes + +- **SC-001**: Graph generation completes in < 5 seconds for applications with up to 50 resources +- **SC-002**: Generated graphs are 100% deterministic (identical input produces byte-identical output) +- **SC-003**: Graph diff correctly identifies all added, removed, and modified resources with zero false positives +- **SC-004**: GitHub Action posts PR comments within 60 seconds of workflow trigger +- **SC-005**: Markdown output (including embedded Mermaid diagram) renders correctly in GitHub without manual formatting +- **SC-006**: Git enrichment adds < 2 seconds overhead for repositories with up to 1000 commits +- **SC-007**: System handles Bicep files up to 5000 lines without performance degradation +- **SC-008**: Error messages include actionable guidance in 100% of failure cases + +--- + +## Testing Requirements *(per Constitution Principle IV)* + +This feature MUST include comprehensive testing across the testing pyramid: + +### Unit Tests +- Test individual graph parsing functions in isolation +- Test git metadata extraction logic +- Test output formatters (JSON, Markdown with embedded Mermaid) with known inputs +- Test error handling for malformed Bicep files +- All unit tests runnable with `make test` without external dependencies + +### Integration Tests +- Test Bicep CLI integration for file parsing +- Test git operations (blame, log) with real git repositories +- Test module resolution across multiple Bicep files + +### Functional Tests +- End-to-end test: Bicep file → graph generation → output validation +- Test GitHub Action in a real PR workflow +- Test graph diff accuracy with known before/after states + +--- + +## Open Questions + +1. **Bicep Compiler Integration**: Should we use the official Bicep CLI for parsing, or implement a lightweight parser? Trade-off: accuracy vs. dependency management. **Recommendation**: Use official Bicep CLI per Constitution Principle VII (Simplicity Over Cleverness). + +2. ~~**Graph Storage**: Should generated graphs be committed to the repo (e.g., `app-graph.json`)? Trade-off: visibility vs. repo noise.~~ **RESOLVED (Initial Implementation)**: Graphs are committed to `.radius/app-graph.json`. This enables lightweight GitHub Action (reads from git history, no tooling required) and provides auditable graph evolution. **Future Evolution**: External storage backends (e.g., SQLite, cloud databases) could be supported for scenarios requiring graph queries across repositories, historical analytics, or enterprise-scale graph management. + +3. ~~**GitHub App vs Action**: Should the PR integration be a GitHub Action (user-managed) or a GitHub App (centrally managed)? Trade-off: flexibility vs. ease of setup.~~ **RESOLVED**: GitHub Action. Fork-friendly, no installation approval required, aligns with existing Radius workflow patterns, supports incremental adoption. + +4. ~~**Diff Visualization**: What's the preferred format for showing diffs in PR comments—table-based, Mermaid side-by-side, or unified text diff?~~ **RESOLVED**: Table + Mermaid diagrams. Change table shows added/removed/modified resources with details; before/after Mermaid diagrams provide visual topology comparison. Both render natively in GitHub. + +5. ~~**Parameter Handling**: How should we handle Bicep parameters without a params file—use defaults, require params, or mark as "unknown"?~~ **RESOLVED**: Require params file. If Bicep has required parameters (no defaults) but no `--parameters` flag is provided, fail with a clear error message listing the missing parameters. + +--- + +## Cross-Repository Impact *(per Constitution Principle XVII)* + +This feature may affect multiple Radius repositories: + +| Repository | Impact | +|------------|--------| +| `radius` | CLI implementation (`pkg/cli/cmd/app/graph/`), core graph logic | +| `docs` | User documentation for new CLI commands, GitHub Action setup guide | +| `design-notes` | This specification and implementation plan | + +Coordinate changes across repositories per Constitution guidance on polyglot project coherence. \ No newline at end of file diff --git a/specs/003-git-app-graph-storage/tasks.md b/specs/003-git-app-graph-storage/tasks.md new file mode 100644 index 0000000000..0e946ed2f9 --- /dev/null +++ b/specs/003-git-app-graph-storage/tasks.md @@ -0,0 +1,323 @@ +# Tasks: Git App Graph Preview + +**Input**: Design documents from `/specs/001-git-app-graph-preview/` +**Prerequisites**: plan.md ✓, spec.md ✓, research.md ✓, data-model.md ✓, contracts/ ✓, quickstart.md ✓ + +**Tests**: Included per spec.md "Testing Requirements" section - comprehensive testing across the testing pyramid. + +**Organization**: Tasks grouped by user story for independent implementation and testing. + +## Format: `[ID] [P?] [Story] Description` + +- **[P]**: Can run in parallel (different files, no dependencies) +- **[Story]**: Which user story this task belongs to (US1, US2, US3, US4, US5, US6) +- Exact file paths for radius repository + +--- + +## Phase 1: Setup (Shared Infrastructure) + +**Purpose**: Project initialization and scaffolding for the static graph feature + +- [ ] T001 Create package structure `pkg/cli/bicep/` with package doc in pkg/cli/bicep/doc.go +- [ ] T002 Create package structure `pkg/cli/git/` with package doc in pkg/cli/git/doc.go +- [ ] T003 [P] Add new test directories: test/unit/cli/graph/, test/integration/cli/graph/, test/functional/cli/graph/ +- [ ] T004 [P] Define AppGraph types in pkg/corerp/api/v20231001preview/appgraph_static_types.go +- [ ] T005 [P] Define GraphDiff types in pkg/corerp/api/v20231001preview/appgraph_diff_types.go +- [ ] T006 Add JSON schema validation helpers in pkg/cli/output/json.go (deterministic key ordering) + +--- + +## Phase 2: Foundational (Blocking Prerequisites) + +**Purpose**: Core infrastructure required by ALL user stories + +**⚠️ CRITICAL**: No user story work can begin until this phase is complete + +- [ ] T007 Implement Bicep CLI executor in pkg/cli/bicep/executor.go (wraps `bicep build --stdout`) +- [ ] T008 Implement ARM JSON parser interface in pkg/cli/bicep/parser.go (extract resources array) +- [ ] T009 Implement resource extraction from ARM JSON in pkg/cli/bicep/extractor.go +- [ ] T010 Implement connection detection from resource properties in pkg/cli/bicep/connections.go +- [ ] T011 [P] Implement source hash computation (SHA256 of input files) in pkg/cli/bicep/hash.go +- [ ] T012 [P] Add Bicep file detection logic (`.bicep` extension) in pkg/cli/cmd/app/graph/detect.go +- [ ] T013 Unit tests for Bicep executor in test/unit/cli/graph/executor_test.go +- [ ] T014 Unit tests for ARM JSON parser in test/unit/cli/graph/parser_test.go + +**Checkpoint**: Foundation ready - user story implementation can begin + +--- + +## Phase 3: User Story 1 - Generate App Graph from Bicep Files (Priority: P1) 🎯 MVP + +**Goal**: Enable `rad app graph app.bicep` to generate a JSON app graph from Bicep files without deployment + +**Independent Test**: Run CLI against sample Bicep file, verify JSON output contains expected resources and connections + +### Tests for User Story 1 + +- [ ] T015 [P] [US1] Unit test for graph generation with valid Bicep in test/unit/cli/graph/static_test.go +- [ ] T016 [P] [US1] Unit test for error handling with invalid Bicep in test/unit/cli/graph/static_errors_test.go +- [ ] T017 [P] [US1] Unit test for module resolution in test/unit/cli/graph/modules_test.go +- [ ] T018 [US1] Integration test with real Bicep CLI in test/integration/cli/graph/bicep_integration_test.go + +### Implementation for User Story 1 + +- [ ] T019 [US1] Implement static graph generator in pkg/cli/cmd/app/graph/static.go +- [ ] T020 [US1] Add file input detection to existing graph.go entry point in pkg/cli/cmd/app/graph/graph.go +- [ ] T021 [US1] Implement module resolution (transitive Bicep imports) in pkg/cli/bicep/modules.go +- [ ] T022 [US1] Add parameter file support (`--parameters`) in pkg/cli/cmd/app/graph/params.go +- [ ] T023 [US1] Implement required parameter validation (fail if missing) in pkg/cli/bicep/params.go +- [ ] T024 [US1] Add Radius Bicep extension type detection in pkg/cli/bicep/radius_types.go +- [ ] T025 [US1] Implement error handling with line/column info in pkg/cli/bicep/errors.go +- [ ] T026 [US1] Add logging for graph generation operations in pkg/cli/cmd/app/graph/static.go + +**Checkpoint**: `rad app graph app.bicep` generates valid JSON graph from Bicep files + +--- + +## Phase 4: User Story 2 - Export Graph as Diff-Friendly Format (Priority: P1) + +**Goal**: Output deterministic JSON to `.radius/app-graph.json` and optional Markdown with Mermaid diagrams + +**Independent Test**: Generate graph twice from identical input, verify byte-identical output; verify Markdown renders in GitHub + +### Tests for User Story 2 + +- [ ] T027 [P] [US2] Unit test for deterministic JSON output in test/unit/cli/graph/json_determinism_test.go +- [ ] T028 [P] [US2] Unit test for Mermaid diagram generation in test/unit/cli/graph/mermaid_test.go +- [ ] T029 [P] [US2] Unit test for Markdown table formatting in test/unit/cli/graph/markdown_test.go +- [ ] T030 [US2] Integration test for file output paths in test/integration/cli/graph/output_test.go + +### Implementation for User Story 2 + +- [ ] T031 [US2] Implement deterministic JSON serializer in pkg/cli/output/deterministic_json.go +- [ ] T032 [US2] Implement default output path (`.radius/app-graph.json`) in pkg/cli/cmd/app/graph/output.go +- [ ] T033 [US2] Add `--stdout` flag for stdout output in pkg/cli/cmd/app/graph/flags.go +- [ ] T034 [US2] Add `-o` flag for custom output path in pkg/cli/cmd/app/graph/flags.go +- [ ] T035 [US2] Implement Mermaid diagram generator in pkg/cli/output/mermaid.go +- [ ] T036 [US2] Implement Markdown table formatter in pkg/cli/output/markdown.go +- [ ] T037 [US2] Add `--format markdown` flag (generates both JSON + Markdown) in pkg/cli/cmd/app/graph/flags.go +- [ ] T038 [US2] Integrate display.go with new output formatters in pkg/cli/cmd/app/graph/display.go +- [ ] T039 [US2] Add Mermaid shape mapping (containers→rectangles, gateways→diamonds, databases→cylinders) in pkg/cli/output/mermaid.go + +**Checkpoint**: Deterministic JSON output + optional Markdown with Mermaid diagrams working + +--- + +## Phase 5: User Story 3 - Git Metadata Enrichment (Priority: P2) + +**Goal**: Automatically enrich each resource with git commit information (SHA, author, date, message) + +**Independent Test**: Generate graph in git repo, verify each resource has git metadata; verify `--no-git` skips enrichment + +### Tests for User Story 3 + +- [ ] T040 [P] [US3] Unit test for git blame parsing in test/unit/cli/graph/git_blame_test.go +- [ ] T041 [P] [US3] Unit test for git log parsing in test/unit/cli/graph/git_log_test.go +- [ ] T042 [P] [US3] Unit test for uncommitted file detection in test/unit/cli/graph/git_uncommitted_test.go +- [ ] T043 [US3] Integration test with real git repository in test/integration/cli/graph/git_integration_test.go + +### Implementation for User Story 3 + +- [ ] T044 [US3] Implement git repository detection in pkg/cli/git/repo.go +- [ ] T045 [US3] Implement git blame executor in pkg/cli/git/blame.go +- [ ] T046 [US3] Implement git log metadata extraction in pkg/cli/git/log.go +- [ ] T047 [US3] Implement per-resource git info enrichment in pkg/cli/git/metadata.go +- [ ] T048 [US3] Add uncommitted changes detection in pkg/cli/git/status.go +- [ ] T049 [US3] Add `--no-git` flag for faster execution in pkg/cli/cmd/app/graph/flags.go +- [ ] T050 [US3] Handle non-git directories gracefully in pkg/cli/git/metadata.go +- [ ] T051 [US3] Handle shallow clones (mark as "history unavailable") in pkg/cli/git/blame.go +- [ ] T052 [US3] Add linked commit SHA in Markdown output in pkg/cli/output/markdown.go + +**Checkpoint**: Graph resources include git metadata by default; `--no-git` works + +--- + +## Phase 6: User Story 4 - GitHub Action for PR Graph Diff (Priority: P2) + +**Goal**: GitHub Action reads committed JSON from git history and posts diff comments on PRs + +**Independent Test**: Create PR with graph changes, verify Action posts comment with change table and Mermaid diagrams + +### Tests for User Story 4 + +- [ ] T053 [P] [US4] Unit test for JSON-to-JSON diff computation in test/unit/cli/graph/diff_test.go +- [ ] T054 [P] [US4] Unit test for diff summary generation in test/unit/cli/graph/diff_summary_test.go +- [ ] T055 [US4] Unit test for diff Markdown rendering in test/unit/cli/graph/diff_render_test.go + +### Implementation for User Story 4 + +- [ ] T056 [US4] Implement graph diff computation in pkg/cli/cmd/app/graph/diff.go +- [ ] T057 [US4] Implement resource comparison (by ID) in pkg/cli/cmd/app/graph/diff.go +- [ ] T058 [US4] Implement connection comparison (by source+target tuple) in pkg/cli/cmd/app/graph/diff.go +- [ ] T059 [US4] Implement diff summary statistics in pkg/cli/cmd/app/graph/diff.go +- [ ] T060 [US4] Implement diff Markdown renderer (change table + before/after Mermaid) in pkg/cli/output/diff_markdown.go +- [ ] T061 [US4] Create GitHub Action definition in actions/app-graph-diff/action.yml +- [ ] T062 [US4] Implement diff computation shell script in actions/app-graph-diff/diff.sh +- [ ] T063 [US4] Implement Mermaid rendering for Action in actions/app-graph-diff/render.js +- [ ] T064 [US4] Add PR comment create-or-update logic in actions/app-graph-diff/comment.sh +- [ ] T065 [US4] Add monorepo support (glob `**/.radius/app-graph.json`) in actions/app-graph-diff/detect.sh +- [ ] T066 [US4] Add staleness validation (compare committed vs regenerated) in actions/app-graph-diff/validate.sh +- [ ] T067 [US4] Add `push` trigger support for baseline tracking in actions/app-graph-diff/action.yml + +**Checkpoint**: GitHub Action posts diff comments on PRs with graph changes + +--- + +## Phase 7: User Story 5 - Historical Graph Timeline (Priority: P3) + +**Goal**: Enable `rad app graph history` to show graph evolution across commits + +**Independent Test**: Run history command on repo with multiple commits, verify timeline shows graph changes + +### Tests for User Story 5 + +- [ ] T068 [P] [US5] Unit test for timeline generation in test/unit/cli/graph/history_test.go +- [ ] T069 [US5] Integration test with multi-commit git history in test/integration/cli/graph/history_integration_test.go + +### Implementation for User Story 5 + +- [ ] T070 [US5] Implement `rad app graph history` subcommand in pkg/cli/cmd/app/graph/history.go +- [ ] T071 [US5] Add `--commits N` flag in pkg/cli/cmd/app/graph/history.go +- [ ] T072 [US5] Implement `--at ` flag for graph at specific commit in pkg/cli/cmd/app/graph/flags.go +- [ ] T073 [US5] Implement `rad app graph diff --from X --to Y` subcommand in pkg/cli/cmd/app/graph/diff.go +- [ ] T074 [US5] Implement commit range iteration in pkg/cli/git/history.go +- [ ] T075 [US5] Implement change summary per commit in pkg/cli/cmd/app/graph/history.go + +**Checkpoint**: Historical timeline and commit-specific graph generation working + +--- + +## Phase 8: User Story 6 - Environment-Resolved Graph (Priority: P3) + +**Goal**: Enable `rad app graph --environment` to show resolved recipe types instead of abstract Radius types + +**Independent Test**: Generate graph with environment flag, verify abstract types resolve to concrete infrastructure types + +### Tests for User Story 6 + +- [ ] T076 [P] [US6] Unit test for recipe type resolution in test/unit/cli/graph/resolve_test.go +- [ ] T077 [US6] Integration test with Radius environment in test/integration/cli/graph/environment_test.go + +### Implementation for User Story 6 + +- [ ] T078 [US6] Add `--environment` flag in pkg/cli/cmd/app/graph/flags.go +- [ ] T079 [US6] Implement environment connection in pkg/cli/cmd/app/graph/environment.go +- [ ] T080 [US6] Implement recipe lookup from environment in pkg/cli/cmd/app/graph/recipes.go +- [ ] T081 [US6] Implement portable type → concrete type resolution in pkg/cli/cmd/app/graph/resolve.go +- [ ] T082 [US6] Add "no recipe bound" annotation for unbound types in pkg/cli/cmd/app/graph/resolve.go +- [ ] T083 [US6] Integrate resolved types into graph output in pkg/cli/cmd/app/graph/static.go + +**Checkpoint**: Environment-resolved graphs show concrete infrastructure types + +--- + +## Phase 9: Polish & Cross-Cutting Concerns + +**Purpose**: Documentation, validation, and refinements + +- [ ] T084 [P] Update pkg/cli/cmd/app/graph/README.md with new command documentation +- [ ] T085 [P] Add godoc comments to all exported types in pkg/cli/bicep/ +- [ ] T086 [P] Add godoc comments to all exported types in pkg/cli/git/ +- [ ] T087 Run golangci-lint and fix any issues in new code +- [ ] T088 [P] Create functional E2E test in test/functional/cli/graph/e2e_test.go +- [ ] T089 Validate quickstart.md scenarios work end-to-end +- [ ] T090 Update CHANGELOG.md with new feature description +- [ ] T091 [P] Create GitHub Action README in actions/app-graph-diff/README.md + +--- + +## Dependencies & Execution Order + +### Phase Dependencies + +- **Setup (Phase 1)**: No dependencies - can start immediately +- **Foundational (Phase 2)**: Depends on Setup - BLOCKS all user stories +- **US1 (Phase 3)**: Depends on Foundational - First MVP +- **US2 (Phase 4)**: Depends on Foundational - Second MVP capability +- **US3 (Phase 5)**: Depends on US1/US2 working - Enrichment layer +- **US4 (Phase 6)**: Depends on US2 (diff-friendly output) + US3 (git metadata) +- **US5 (Phase 7)**: Depends on US3 (git integration) +- **US6 (Phase 8)**: Can start after US1 - independent branch +- **Polish (Phase 9)**: After all desired user stories complete + +### User Story Dependencies + +``` +Foundational (Phase 2) + │ + ├──────────────┬──────────────┐ + ▼ ▼ ▼ + US1 (P1) US2 (P1) US6 (P3) + │ │ (independent) + └──────┬───────┘ + ▼ + US3 (P2) + │ + ┌──────┴──────┐ + ▼ ▼ + US4 (P2) US5 (P3) +``` + +### Parallel Opportunities + +**Phase 1 (Setup)**: T001, T002 sequential; T003-T006 all parallel +**Phase 2 (Foundational)**: T007-T010 sequential; T011-T012 parallel; T013-T014 parallel +**Each User Story**: All tests marked [P] can run in parallel; implementation tasks mostly sequential within story + +--- + +## Parallel Example: User Story 1 Tests + +```bash +# Launch all US1 tests in parallel: +Task: "T015 [P] [US1] Unit test for graph generation with valid Bicep" +Task: "T016 [P] [US1] Unit test for error handling with invalid Bicep" +Task: "T017 [P] [US1] Unit test for module resolution" + +# Then run integration test: +Task: "T018 [US1] Integration test with real Bicep CLI" +``` + +--- + +## Implementation Strategy + +### MVP First (US1 + US2 = P1 Stories) + +1. Complete Phase 1: Setup +2. Complete Phase 2: Foundational (CRITICAL) +3. Complete Phase 3: User Story 1 (core graph generation) +4. Complete Phase 4: User Story 2 (diff-friendly output) +5. **STOP and VALIDATE**: Run quickstart.md scenarios locally +6. Deploy CLI changes for developer feedback + +### Git Integration (P2 Stories) + +1. Add US3: Git metadata enrichment +2. Add US4: GitHub Action for PR diffs +3. **VALIDATE**: Test in real PR workflow + +### Advanced Features (P3 Stories) + +1. Add US5: Historical timeline (optional) +2. Add US6: Environment resolution (optional) +3. Final polish phase + +### Parallel Team Strategy + +With multiple developers: +- **Developer A**: Phase 1 → Phase 2 → US1 → US3 → US5 +- **Developer B**: Phase 1 (parallel) → US2 → US4 (GitHub Action) +- **Developer C**: US6 (can start after Phase 2) + +--- + +## Notes + +- Radius repository is at `../radius` relative to design-notes +- All Go code must pass `golangci-lint` (Constitution Principle II) +- All exported types need godoc comments (NFR-002) +- JSON output must be deterministic (FR-004, SC-002) +- Existing `rad app graph ` behavior must not change (FR-003) +- GitHub Action is lightweight (reads JSON only, no Bicep tooling) From 7ae48e56e355378147b532e5e49acfa9f816488e Mon Sep 17 00:00:00 2001 From: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com> Date: Mon, 11 May 2026 12:13:32 -0400 Subject: [PATCH 2/2] remove spec files Signed-off-by: Brooke Hamilton <45323234+brooke-hamilton@users.noreply.github.com> --- .../checklists/requirements.md | 57 --- .../contracts/app-graph-schema.yaml | 300 ------------ specs/003-git-app-graph-storage/data-model.md | 410 ----------------- specs/003-git-app-graph-storage/plan.md | 227 --------- specs/003-git-app-graph-storage/quickstart.md | 268 ----------- specs/003-git-app-graph-storage/research.md | 239 ---------- specs/003-git-app-graph-storage/spec.md | 430 ------------------ specs/003-git-app-graph-storage/tasks.md | 323 ------------- 8 files changed, 2254 deletions(-) delete mode 100644 specs/003-git-app-graph-storage/checklists/requirements.md delete mode 100644 specs/003-git-app-graph-storage/contracts/app-graph-schema.yaml delete mode 100644 specs/003-git-app-graph-storage/data-model.md delete mode 100644 specs/003-git-app-graph-storage/plan.md delete mode 100644 specs/003-git-app-graph-storage/quickstart.md delete mode 100644 specs/003-git-app-graph-storage/research.md delete mode 100644 specs/003-git-app-graph-storage/spec.md delete mode 100644 specs/003-git-app-graph-storage/tasks.md diff --git a/specs/003-git-app-graph-storage/checklists/requirements.md b/specs/003-git-app-graph-storage/checklists/requirements.md deleted file mode 100644 index 39ab93ebbd..0000000000 --- a/specs/003-git-app-graph-storage/checklists/requirements.md +++ /dev/null @@ -1,57 +0,0 @@ -# Specification Quality Checklist: Git App Graph Preview - -**Purpose**: Validate specification completeness and quality before proceeding to planning -**Created**: January 30, 2026 -**Last Validated**: January 30, 2026 -**Feature**: [spec.md](../spec.md) - -## Content Quality - -- [x] No implementation details (languages, frameworks, APIs) -- [x] Focused on user value and business needs -- [x] Written for non-technical stakeholders -- [x] All mandatory sections completed - -## Requirement Completeness - -- [x] No [NEEDS CLARIFICATION] markers remain -- [x] Requirements are testable and unambiguous -- [x] Success criteria are measurable -- [x] Success criteria are technology-agnostic (no implementation details) -- [x] All acceptance scenarios are defined -- [x] Edge cases are identified -- [x] Scope is clearly bounded -- [x] Dependencies and assumptions identified - -## Feature Readiness - -- [x] All functional requirements have clear acceptance criteria -- [x] User scenarios cover primary flows -- [x] Feature meets measurable outcomes defined in Success Criteria -- [x] No implementation details leak into specification - -## Notes - -**Validation Summary**: All items passed ✓ - -**Spec Statistics**: -- 6 user stories with clear priorities (P1: 2, P2: 2, P3: 2) -- 17 functional requirements (FR-001 to FR-017) -- 5 non-functional requirements (NFR-001 to NFR-005) -- 8 measurable success criteria (SC-001 to SC-008) -- 6 edge cases documented -- 5 open questions identified for planning phase - -**Notable Additions Since Initial Draft**: -- Problem Statement section added -- Constitution Alignment mapping added -- Testing Requirements section added (per Constitution Principle IV) -- Non-Functional Requirements added (organizational code quality standards) -- Cross-Repository Impact section added -- Open Questions section captures decisions needed - -**Minor Notes (Acceptable)**: -- NFRs reference Go/golangci-lint - acceptable as organizational standards, not implementation choices -- Cross-Repository Impact mentions repo paths - acceptable as planning information - -**Status**: Specification is ready for `/speckit.clarify` or `/speckit.plan` diff --git a/specs/003-git-app-graph-storage/contracts/app-graph-schema.yaml b/specs/003-git-app-graph-storage/contracts/app-graph-schema.yaml deleted file mode 100644 index 15c04bf6e2..0000000000 --- a/specs/003-git-app-graph-storage/contracts/app-graph-schema.yaml +++ /dev/null @@ -1,300 +0,0 @@ -openapi: 3.0.3 -info: - title: Radius App Graph Static API - description: | - Internal API contract for static app graph generation from Bicep files. - This contract defines the JSON schema for app graph output and diff operations. - - Note: This is a CLI-internal contract, not a REST API. The schema defines - the structure of JSON files written by `rad app graph` commands. - version: 1.0.0 - license: - name: Apache 2.0 - url: https://www.apache.org/licenses/LICENSE-2.0 - -paths: {} - -components: - schemas: - AppGraph: - type: object - description: Complete application topology extracted from Bicep files - required: - - metadata - - resources - - connections - properties: - metadata: - $ref: '#/components/schemas/AppGraphMetadata' - resources: - type: array - items: - $ref: '#/components/schemas/AppGraphResource' - connections: - type: array - items: - $ref: '#/components/schemas/AppGraphConnection' - - AppGraphMetadata: - type: object - description: Generation context and provenance information - required: - - generatedAt - - radiusCliVersion - - sourceFiles - - sourceHash - properties: - generatedAt: - type: string - format: date-time - description: UTC timestamp when this graph was generated - example: "2026-01-30T10:15:00Z" - radiusCliVersion: - type: string - description: Version of rad CLI used to generate this graph - example: "0.35.0" - sourceFiles: - type: array - items: - type: string - description: Bicep files that contributed to this graph - example: ["app.bicep", "modules/database.bicep"] - sourceHash: - type: string - pattern: "^sha256:[a-f0-9]{64}$" - description: SHA256 hash of source files for staleness detection - example: "sha256:7d865e959b2466918c9863afca942d0fb89d7c9ac0c99bafc3749504ded97730" - gitCommit: - type: string - description: Current git commit SHA (if in a git repository) - example: "abc123def456" - - AppGraphResource: - type: object - description: A single resource in the application topology - required: - - id - - name - - type - - sourceLocation - properties: - id: - type: string - description: Fully qualified Radius resource ID - pattern: "^/planes/radius/.+/providers/.+/.+" - example: "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend" - name: - type: string - description: Human-readable resource name - example: "frontend" - type: - type: string - description: Resource type - example: "Applications.Core/containers" - sourceLocation: - $ref: '#/components/schemas/SourceLocation' - gitInfo: - $ref: '#/components/schemas/GitInfo' - properties: - type: object - additionalProperties: true - description: Type-specific resource configuration - - SourceLocation: - type: object - description: Bicep source file and line where a resource is defined - required: - - file - - line - properties: - file: - type: string - description: Path to Bicep file (relative to repo root) - example: "app.bicep" - line: - type: integer - minimum: 1 - description: 1-based line number where resource begins - example: 12 - module: - type: string - description: Module path if from an imported module - example: "modules/database.bicep" - - GitInfo: - type: object - description: Git commit information for a resource - required: - - commitSha - - commitShort - - author - - date - - message - properties: - commitSha: - type: string - pattern: "^[a-f0-9]{40}$" - description: Full commit hash that last modified this resource - example: "abc123def456789012345678901234567890abcd" - commitShort: - type: string - pattern: "^[a-f0-9]{7}$" - description: Abbreviated commit hash for display - example: "abc123d" - author: - type: string - format: email - description: Commit author email - example: "dev@example.com" - date: - type: string - format: date-time - description: Commit timestamp in RFC3339 format - example: "2026-01-29T14:30:00Z" - message: - type: string - description: Commit message (first line only) - example: "Add frontend container" - uncommitted: - type: boolean - description: True if resource has uncommitted changes - default: false - - AppGraphConnection: - type: object - description: Directed edge between two resources - required: - - sourceId - - targetId - - type - properties: - sourceId: - type: string - description: Resource ID where connection originates - example: "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend" - targetId: - type: string - description: Resource ID where connection terminates - example: "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/backend" - type: - type: string - enum: - - connection - - route - - dependsOn - description: Kind of connection - - GraphDiff: - type: object - description: Differences between two app graphs - required: - - addedResources - - removedResources - - modifiedResources - - addedConnections - - removedConnections - - summary - properties: - baseCommit: - type: string - description: Commit SHA of the base graph - example: "abc123" - headCommit: - type: string - description: Commit SHA of the head graph - example: "def456" - addedResources: - type: array - items: - $ref: '#/components/schemas/AppGraphResource' - description: Resources in head but not in base - removedResources: - type: array - items: - $ref: '#/components/schemas/AppGraphResource' - description: Resources in base but not in head - modifiedResources: - type: array - items: - $ref: '#/components/schemas/ResourceDiff' - description: Resources with changed properties - addedConnections: - type: array - items: - $ref: '#/components/schemas/AppGraphConnection' - description: Connections in head but not in base - removedConnections: - type: array - items: - $ref: '#/components/schemas/AppGraphConnection' - description: Connections in base but not in head - summary: - $ref: '#/components/schemas/DiffSummary' - - ResourceDiff: - type: object - description: Changes to a single resource - required: - - id - - name - - type - - changedProperties - properties: - id: - type: string - description: Resource ID - name: - type: string - description: Resource name - type: - type: string - description: Resource type - changedProperties: - type: array - items: - $ref: '#/components/schemas/PropertyChange' - - PropertyChange: - type: object - description: Single property modification - required: - - path - properties: - path: - type: string - description: JSON path to changed property - example: "properties.container.image" - oldValue: - description: Value in base graph - newValue: - description: Value in head graph - - DiffSummary: - type: object - description: Aggregate statistics for the diff - required: - - totalChanges - - resourcesAdded - - resourcesRemoved - - resourcesModified - - connectionsAdded - - connectionsRemoved - properties: - totalChanges: - type: integer - minimum: 0 - resourcesAdded: - type: integer - minimum: 0 - resourcesRemoved: - type: integer - minimum: 0 - resourcesModified: - type: integer - minimum: 0 - connectionsAdded: - type: integer - minimum: 0 - connectionsRemoved: - type: integer - minimum: 0 diff --git a/specs/003-git-app-graph-storage/data-model.md b/specs/003-git-app-graph-storage/data-model.md deleted file mode 100644 index cc35aac69a..0000000000 --- a/specs/003-git-app-graph-storage/data-model.md +++ /dev/null @@ -1,410 +0,0 @@ -# Data Model: Git App Graph Preview - -**Feature Branch**: `001-git-app-graph-preview` -**Date**: February 4, 2026 - -## Core Entities - -### AppGraph - -The root container for all graph data. This is the primary structure serialized to JSON. - -```go -// AppGraph represents the complete application topology extracted from Bicep files. -// This is the primary output of the `rad app graph ` command. -type AppGraph struct { - // Metadata contains generation context and provenance information - Metadata AppGraphMetadata `json:"metadata"` - - // Resources contains all resource nodes in the application - Resources []AppGraphResource `json:"resources"` - - // Connections contains all edges between resources - Connections []AppGraphConnection `json:"connections"` -} -``` - -### AppGraphMetadata - -Provenance and staleness detection information. - -```go -// AppGraphMetadata contains generation context for the app graph. -type AppGraphMetadata struct { - // GeneratedAt is the UTC timestamp when this graph was generated - GeneratedAt time.Time `json:"generatedAt"` - - // RadiusCliVersion is the version of rad CLI used to generate this graph - RadiusCliVersion string `json:"radiusCliVersion"` - - // SourceFiles lists all Bicep files that contributed to this graph - SourceFiles []string `json:"sourceFiles"` - - // SourceHash is a SHA256 hash of all source files for staleness detection - SourceHash string `json:"sourceHash"` - - // GitCommit is the current git commit SHA (if in a git repository) - GitCommit string `json:"gitCommit,omitempty"` -} -``` - -### AppGraphResource - -A single resource node in the application graph. - -```go -// AppGraphResource represents a single resource in the application topology. -type AppGraphResource struct { - // ID is the fully qualified Radius resource ID - // Format: /planes/radius/local/resourceGroups/{rg}/providers/{type}/{name} - ID string `json:"id"` - - // Name is the human-readable resource name - Name string `json:"name"` - - // Type is the resource type (e.g., Applications.Core/containers) - Type string `json:"type"` - - // SourceLocation indicates where this resource is defined - SourceLocation SourceLocation `json:"sourceLocation"` - - // GitInfo contains git metadata for this resource (optional) - GitInfo *GitInfo `json:"gitInfo,omitempty"` - - // Properties contains type-specific resource configuration - // Stored as map for flexibility across resource types - Properties map[string]any `json:"properties,omitempty"` -} -``` - -### SourceLocation - -Source file tracking for each resource. - -```go -// SourceLocation indicates the Bicep source file and line where a resource is defined. -type SourceLocation struct { - // File is the path to the Bicep file (relative to repo root) - File string `json:"file"` - - // Line is the 1-based line number where the resource begins - Line int `json:"line"` - - // Module is the module path if this resource is from an imported module - Module string `json:"module,omitempty"` -} -``` - -### GitInfo - -Git metadata for a resource, populated via `git blame`. - -```go -// GitInfo contains git commit information for a resource. -type GitInfo struct { - // CommitSHA is the full commit hash that last modified this resource - CommitSHA string `json:"commitSha"` - - // CommitShort is the abbreviated commit hash for display - CommitShort string `json:"commitShort"` - - // Author is the commit author email - Author string `json:"author"` - - // Date is the commit timestamp in RFC3339 format - Date time.Time `json:"date"` - - // Message is the commit message (first line only) - Message string `json:"message"` - - // Uncommitted indicates this resource has uncommitted changes - Uncommitted bool `json:"uncommitted,omitempty"` -} -``` - -### AppGraphConnection - -An edge between two resources representing a dependency or data flow. - -```go -// AppGraphConnection represents a directed edge between two resources. -type AppGraphConnection struct { - // SourceID is the resource ID where the connection originates - SourceID string `json:"sourceId"` - - // TargetID is the resource ID where the connection terminates - TargetID string `json:"targetId"` - - // Type indicates the kind of connection - Type ConnectionType `json:"type"` -} - -// ConnectionType enumerates the kinds of resource connections. -type ConnectionType string - -const ( - // ConnectionTypeConnection represents a direct connection (e.g., container to database) - ConnectionTypeConnection ConnectionType = "connection" - - // ConnectionTypeRoute represents a gateway route to a destination - ConnectionTypeRoute ConnectionType = "route" - - // ConnectionTypeDependsOn represents an explicit dependsOn relationship - ConnectionTypeDependsOn ConnectionType = "dependsOn" -) -``` - -## Diff Entities - -### GraphDiff - -The result of comparing two app graphs. - -```go -// GraphDiff represents the differences between two app graphs. -type GraphDiff struct { - // BaseCommit is the commit SHA of the base graph (optional) - BaseCommit string `json:"baseCommit,omitempty"` - - // HeadCommit is the commit SHA of the head graph (optional) - HeadCommit string `json:"headCommit,omitempty"` - - // AddedResources are resources present in head but not in base - AddedResources []AppGraphResource `json:"addedResources"` - - // RemovedResources are resources present in base but not in head - RemovedResources []AppGraphResource `json:"removedResources"` - - // ModifiedResources are resources with changed properties - ModifiedResources []ResourceDiff `json:"modifiedResources"` - - // AddedConnections are connections present in head but not in base - AddedConnections []AppGraphConnection `json:"addedConnections"` - - // RemovedConnections are connections present in base but not in head - RemovedConnections []AppGraphConnection `json:"removedConnections"` - - // Summary provides a human-readable overview - Summary DiffSummary `json:"summary"` -} - -// ResourceDiff captures changes to a single resource. -type ResourceDiff struct { - // ID is the resource ID (same in both base and head) - ID string `json:"id"` - - // Name is the resource name - Name string `json:"name"` - - // Type is the resource type - Type string `json:"type"` - - // ChangedProperties lists the property paths that changed - ChangedProperties []PropertyChange `json:"changedProperties"` -} - -// PropertyChange describes a single property modification. -type PropertyChange struct { - // Path is the JSON path to the changed property (e.g., "properties.container.image") - Path string `json:"path"` - - // OldValue is the value in the base graph - OldValue any `json:"oldValue,omitempty"` - - // NewValue is the value in the head graph - NewValue any `json:"newValue,omitempty"` -} - -// DiffSummary provides aggregate statistics for the diff. -type DiffSummary struct { - TotalChanges int `json:"totalChanges"` - ResourcesAdded int `json:"resourcesAdded"` - ResourcesRemoved int `json:"resourcesRemoved"` - ResourcesModified int `json:"resourcesModified"` - ConnectionsAdded int `json:"connectionsAdded"` - ConnectionsRemoved int `json:"connectionsRemoved"` -} -``` - -## Entity Relationships - -``` -┌─────────────────┐ -│ AppGraph │ -├─────────────────┤ -│ Metadata │──────┐ -│ Resources[] │──┐ │ -│ Connections[] │ │ │ -└─────────────────┘ │ │ - │ │ - ┌───────────────┘ │ - ▼ ▼ -┌─────────────────┐ ┌──────────────────┐ -│ AppGraphResource│ │ AppGraphMetadata │ -├─────────────────┤ ├──────────────────┤ -│ ID │ │ GeneratedAt │ -│ Name │ │ RadiusCliVersion │ -│ Type │ │ SourceFiles[] │ -│ SourceLocation │──┐ SourceHash │ -│ GitInfo? │ │ GitCommit? │ -│ Properties │ └──────────────────┘ -└─────────────────┘ - │ - ┌────┴────┐ - ▼ ▼ -┌──────────┐ ┌─────────┐ -│SourceLoc │ │ GitInfo │ -├──────────┤ ├─────────┤ -│ File │ │CommitSHA│ -│ Line │ │Author │ -│ Module? │ │Date │ -└──────────┘ │Message │ - └─────────┘ - -┌────────────────────┐ -│ AppGraphConnection │ -├────────────────────┤ -│ SourceID │───────► AppGraphResource.ID -│ TargetID │───────► AppGraphResource.ID -│ Type │ -└────────────────────┘ -``` - -## Validation Rules - -### AppGraph -- `Metadata.GeneratedAt` MUST be a valid UTC timestamp -- `Metadata.SourceFiles` MUST contain at least one file -- `Metadata.SourceHash` MUST be a valid SHA256 hash (64 hex chars) -- `Resources` MAY be empty for an empty Bicep file - -### AppGraphResource -- `ID` MUST be a valid Radius resource ID format -- `Name` MUST be non-empty -- `Type` MUST be a recognized resource type or follow ARM type pattern -- `SourceLocation.File` MUST be a valid relative file path -- `SourceLocation.Line` MUST be >= 1 - -### AppGraphConnection -- `SourceID` MUST reference an existing resource ID -- `TargetID` MUST reference an existing resource ID -- `Type` MUST be one of the defined ConnectionType values - -### GraphDiff -- All resource references in Added/Removed/Modified MUST be valid -- `Summary` counts MUST match the actual array lengths - -## State Transitions - -Resources can be in the following states relative to git: - -``` -┌──────────────┐ -│ Uncommitted │ ───(git add)───► ┌──────────┐ -└──────────────┘ │ Staged │ - └──────────┘ - │ - (git commit) - │ - ▼ -┌──────────────┐ ┌──────────┐ -│ Modified │ ◄──(edit file)──│Committed │ -└──────────────┘ └──────────┘ - │ - (git add + commit) - │ - ▼ - ┌──────────┐ - │Committed │ (new SHA) - └──────────┘ -``` - -Graph states: -- **Current**: Generated from current working directory files -- **Committed**: Exists in `.radius/app-graph.json` in git history -- **Stale**: Committed graph doesn't match current Bicep files (detected via sourceHash) - -## JSON Schema Example - -```json -{ - "metadata": { - "generatedAt": "2026-01-30T10:15:00Z", - "radiusCliVersion": "0.35.0", - "sourceFiles": [ - "app.bicep", - "modules/database.bicep" - ], - "sourceHash": "sha256:7d865e959b2466918c9863afca942d0fb89d7c9ac0c99bafc3749504ded97730", - "gitCommit": "abc123def456" - }, - "resources": [ - { - "id": "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend", - "name": "frontend", - "type": "Applications.Core/containers", - "sourceLocation": { - "file": "app.bicep", - "line": 12 - }, - "gitInfo": { - "commitSha": "abc123def456789012345678901234567890abcd", - "commitShort": "abc123d", - "author": "dev@example.com", - "date": "2026-01-29T14:30:00Z", - "message": "Add frontend container" - }, - "properties": { - "container": { - "image": "myapp/frontend:v1.2.3" - } - } - }, - { - "id": "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/backend", - "name": "backend", - "type": "Applications.Core/containers", - "sourceLocation": { - "file": "app.bicep", - "line": 28 - }, - "gitInfo": { - "commitSha": "def456abc789012345678901234567890abcdef01", - "commitShort": "def456a", - "author": "dev@example.com", - "date": "2026-01-28T09:15:00Z", - "message": "Add backend service" - } - }, - { - "id": "/planes/radius/local/resourceGroups/default/providers/Applications.Datastores/redisCaches/cache", - "name": "cache", - "type": "Applications.Datastores/redisCaches", - "sourceLocation": { - "file": "modules/database.bicep", - "line": 5, - "module": "modules/database.bicep" - }, - "gitInfo": { - "commitSha": "789abc012def345678901234567890abcdef0123", - "commitShort": "789abc0", - "author": "ops@example.com", - "date": "2026-01-27T16:45:00Z", - "message": "Add Redis cache for session storage" - } - } - ], - "connections": [ - { - "sourceId": "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/frontend", - "targetId": "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/backend", - "type": "connection" - }, - { - "sourceId": "/planes/radius/local/resourceGroups/default/providers/Applications.Core/containers/backend", - "targetId": "/planes/radius/local/resourceGroups/default/providers/Applications.Datastores/redisCaches/cache", - "type": "connection" - } - ] -} -``` diff --git a/specs/003-git-app-graph-storage/plan.md b/specs/003-git-app-graph-storage/plan.md deleted file mode 100644 index 590b591ee5..0000000000 --- a/specs/003-git-app-graph-storage/plan.md +++ /dev/null @@ -1,227 +0,0 @@ -# Implementation Plan: Git App Graph Preview - -**Branch**: `001-git-app-graph-preview` | **Date**: February 4, 2026 | **Spec**: [spec.md](./spec.md) -**Input**: Feature specification from `/specs/001-git-app-graph-preview/spec.md` -**User Prompt**: /speckit.plan Leverage the existing Radius technology stack for CLI, etc. as well as the application and environment, and resource definitions (https://github.com/radius-project/radius, https://docs.radapp.io/). Data structures and general diff constructs should use Git principles while specific integrations and renderings in the GitHub UI should use their technologies (e.g. GitHub Actions, GitHub Apps, etc.) such that if we were to build similar integrations in the future (for example, with GitLab), our design choices today won't limit us in the future. - -## Summary - -Enable **static app graph generation** from Bicep files without deployment, enriched with **git changelog metadata**, to support visualization and diffing in GitHub PR workflows. The implementation extends the existing `rad app graph` command to accept Bicep file input, generates deterministic JSON output suitable for version control, and provides a lightweight GitHub Action for PR diff visualization. - -**Technical Approach** (from [research.md](./research.md)): -- Use official Bicep CLI to compile `.bicep` → ARM JSON, then extract resources and connections -- Extend existing CLI patterns in `pkg/cli/cmd/app/graph/` with file-based input detection -- Add git metadata via shell exec (`git blame`, `git log`) for source tracking -- Output to `.radius/app-graph.json` (committed artifact model) -- GitHub Action reads JSON from git history, computes diff, posts Mermaid-enhanced PR comments - -## Technical Context - -**Language/Version**: Go 1.21+ (matches existing Radius codebase) -**Primary Dependencies**: -- Cobra (CLI framework, existing) -- Bicep CLI (external, managed via `rad bicep download`) -- Git (external, system requirement) -**Storage**: File-based (`.radius/app-graph.json` committed to git) -**Testing**: `make test` (Go unit tests), `make functional-test` (E2E) -**Target Platform**: Linux, macOS, Windows (cross-platform CLI) -**Project Type**: CLI extension (single project, existing radius repository) -**Performance Goals**: -- Graph generation < 5s for 50 resources (per SC-001) -- Git enrichment adds < 2s for 1000 commits (per SC-006) -**Constraints**: -- Must work without Radius environment (static analysis only) -- Must handle Bicep files up to 5000 lines (per SC-007) -**Scale/Scope**: Applications with up to 100+ resources, graphs committed across all Radius users - -## Constitution Check - -*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.* - -| Principle | Status | Notes | -|-----------|--------|-------| -| **I. API-First Design** | ✅ PASS | JSON schema defined in [contracts/](./contracts/), OpenAPI spec provided | -| **II. Idiomatic Code Standards** | ✅ PASS | Go implementation follows Effective Go; uses existing `pkg/cli/` patterns | -| **III. Multi-Cloud Neutrality** | ✅ PASS | Works with any cloud's Bicep resources; cloud-specific types rendered generically | -| **IV. Testing Pyramid Discipline** | ✅ PASS | Unit tests for parsing/diffing, integration tests for git/Bicep CLI, functional E2E tests | -| **V. Collaboration-Centric Design** | ✅ PASS | Developers preview graphs locally; platform engineers review in PRs | -| **VI. Open Source & Community-First** | ✅ PASS | Design spec in public repo; GitHub Action works in forks | -| **VII. Simplicity Over Cleverness** | ✅ PASS | Uses Bicep CLI (not custom parser); shell exec for git (not libgit2) | -| **VIII. Separation of Concerns** | ✅ PASS | Core diff logic platform-agnostic; GitHub rendering layer separate | -| **IX. Incremental Adoption** | ✅ PASS | New capability; doesn't change existing `rad app graph ` behavior | -| **XIV. Documentation Quality** | ✅ PASS | [quickstart.md](./quickstart.md) follows Diátaxis tutorial pattern | -| **XVII. Polyglot Coherence** | ✅ PASS | Cross-repo impact documented in spec; coordinated with `radius` and `docs` repos | - -**Post-Design Re-Check**: All principles remain satisfied. Platform abstraction in architecture supports future GitLab integration per user requirement. - -## Project Structure - -### Documentation (this feature) - -```text -specs/001-git-app-graph-preview/ -├── spec.md # Feature specification (input) -├── plan.md # This file -├── research.md # Phase 0 output (technology decisions) -├── data-model.md # Phase 1 output (entity definitions) -├── quickstart.md # Phase 1 output (user guide) -├── contracts/ # Phase 1 output (JSON schema) -│ └── app-graph-schema.yaml -└── tasks.md # Phase 2 output (implementation tasks) -``` - -### Source Code (radius repository) - -```text -# radius repository (../radius) -pkg/ -├── cli/ -│ ├── cmd/ -│ │ └── app/ -│ │ └── graph/ -│ │ ├── graph.go # Extended: file input detection -│ │ ├── static.go # NEW: static graph generation -│ │ ├── diff.go # NEW: graph diff computation -│ │ ├── display.go # Extended: Markdown/Mermaid output -│ │ └── graph_test.go # Extended: new test cases -│ ├── bicep/ -│ │ └── parser.go # NEW: ARM JSON → AppGraph extraction -│ └── git/ -│ └── metadata.go # NEW: git blame/log integration -├── corerp/ -│ └── api/ -│ └── v20231001preview/ -│ └── appgraph_types.go # Extended: new types for static graphs - -test/ -├── unit/ -│ └── cli/ -│ └── graph/ # NEW: unit tests for graph generation -├── integration/ -│ └── cli/ -│ └── graph/ # NEW: integration tests with git/Bicep -└── functional/ - └── cli/ - └── graph/ # NEW: E2E test scenarios - -# GitHub Action (separate repo or actions/ folder) -actions/ -└── app-graph-diff/ - ├── action.yml # Action definition - ├── diff.sh # Diff computation script - └── render.js # Markdown/Mermaid rendering -``` - -**Structure Decision**: Extends existing `radius` repository structure. New code follows established patterns in `pkg/cli/`. GitHub Action is a separate deliverable, potentially in its own repo (`radius-project/app-graph-diff-action`). - -## Architecture - -### Component Diagram - -``` -┌─────────────────────────────────────────────────────────────────────────┐ -│ rad CLI │ -├─────────────────────────────────────────────────────────────────────────┤ -│ │ -│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ -│ │ graph.go │ │ static.go │ │ diff.go │ │ -│ │ (entry point) │───▶│ (file input) │───▶│ (comparison) │ │ -│ │ │ │ │ │ │ │ -│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ -│ │ │ │ │ -│ ▼ ▼ ▼ │ -│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ -│ │ Radius API │ │ Bicep CLI │ │ Git CLI │ │ -│ │ (deployed apps) │ │ (compilation) │ │ (metadata) │ │ -│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ -│ │ │ │ -│ ▼ ▼ │ -│ ┌──────────────────┐ ┌──────────────────┐ │ -│ │ parser.go │ │ metadata.go │ │ -│ │ (ARM→AppGraph) │ │ (blame/log) │ │ -│ └──────────────────┘ └──────────────────┘ │ -│ │ │ │ -│ └───────────┬───────────┘ │ -│ ▼ │ -│ ┌──────────────────────┐ │ -│ │ AppGraph (JSON) │ │ -│ │ .radius/app-graph.json │ -│ └──────────────────────┘ │ -└─────────────────────────────────────────────────────────────────────────┘ - -┌─────────────────────────────────────────────────────────────────────────┐ -│ GitHub Action │ -├─────────────────────────────────────────────────────────────────────────┤ -│ │ -│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ -│ │ Checkout │───▶│ diff.sh │───▶│ render.js │ │ -│ │ (base + head) │ │ (JSON compare) │ │ (Mermaid) │ │ -│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ -│ │ │ -│ ▼ │ -│ ┌──────────────────────┐ │ -│ │ PR Comment │ │ -│ │ (create-or-update) │ │ -│ └──────────────────────┘ │ -└─────────────────────────────────────────────────────────────────────────┘ -``` - -### Platform Abstraction (per user requirement) - -``` -┌─────────────────────────────────────────────────────────────────────────┐ -│ Platform-Agnostic Core │ -├─────────────────────────────────────────────────────────────────────────┤ -│ • AppGraph data model (JSON schema) │ -│ • Bicep → ARM → AppGraph parsing │ -│ • Git metadata extraction │ -│ • Diff computation (JSON semantic comparison) │ -│ • Core rendering (Markdown tables, Mermaid diagrams) │ -└─────────────────────────────────────────────────────────────────────────┘ - │ - ┌─────────────────────────┼─────────────────────────┐ - ▼ ▼ ▼ -┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ -│ GitHub Layer │ │ GitLab Layer │ │ CLI Layer │ -│ (Action) │ │ (Future) │ │ (Terminal) │ -├──────────────────┤ ├──────────────────┤ ├──────────────────┤ -│ • PR comments │ │ • MR notes │ │ • stdout │ -│ • Workflow YAML │ │ • CI YAML │ │ • --format │ -│ • Mermaid render│ │ • Mermaid render│ │ • diff command │ -└──────────────────┘ └──────────────────┘ └──────────────────┘ -``` - -## Implementation Phases - -### Phase 1 (P1) - Core Graph Generation -- US1: Generate app graph from Bicep files -- US2: Export graph as diff-friendly JSON/Markdown - -### Phase 2 (P2) - Git & GitHub Integration -- US3: Git metadata enrichment -- US4: GitHub Action for PR diff comments - -### Phase 3 (P3) - Advanced Features -- US5: Historical graph timeline -- US6: Environment-resolved graphs - -## Complexity Tracking - -> No complexity violations identified. - -| Principle | Evaluation | Status | -|-----------|------------|--------| -| Simplicity Over Cleverness | Using Bicep CLI (not custom parser) | ✅ Simple | -| Simplicity Over Cleverness | Shell exec for git (not libgit2) | ✅ Simple | -| Incremental Adoption | Additive to existing CLI; no breaking changes | ✅ Non-disruptive | - -## Generated Artifacts - -- [research.md](./research.md) - Technology decisions and alternatives -- [data-model.md](./data-model.md) - Entity definitions and relationships -- [quickstart.md](./quickstart.md) - User-facing tutorial -- [contracts/app-graph-schema.yaml](./contracts/app-graph-schema.yaml) - OpenAPI JSON schema - -## Next Steps - -Run `/speckit.tasks` to generate actionable implementation tasks from this plan. diff --git a/specs/003-git-app-graph-storage/quickstart.md b/specs/003-git-app-graph-storage/quickstart.md deleted file mode 100644 index 90bb3f65ff..0000000000 --- a/specs/003-git-app-graph-storage/quickstart.md +++ /dev/null @@ -1,268 +0,0 @@ -# Quickstart: Git App Graph Preview - -This guide walks you through generating and using app graphs from Bicep files. - -## Prerequisites - -- [Radius CLI](https://docs.radapp.io/getting-started/install/) (v0.35.0+) -- [Bicep CLI](https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/install) (installed automatically by `rad bicep download`) -- A Bicep file defining a Radius application -- Git repository (optional, for git metadata enrichment) - -## 1. Generate an App Graph - -Given a Bicep file `app.bicep`: - -```bicep -extension radius - -@description('The application name') -param appName string = 'myapp' - -resource app 'Applications.Core/applications@2023-10-01-preview' = { - name: appName - properties: { - environment: environment - } -} - -resource frontend 'Applications.Core/containers@2023-10-01-preview' = { - name: 'frontend' - properties: { - application: app.id - container: { - image: 'myapp/frontend:v1.0.0' - ports: { - web: { containerPort: 3000 } - } - } - connections: { - backend: { source: backend.id } - } - } -} - -resource backend 'Applications.Core/containers@2023-10-01-preview' = { - name: 'backend' - properties: { - application: app.id - container: { - image: 'myapp/backend:v1.0.0' - ports: { - api: { containerPort: 8080 } - } - } - connections: { - cache: { source: cache.id } - } - } -} - -resource cache 'Applications.Datastores/redisCaches@2023-10-01-preview' = { - name: 'cache' - properties: { - application: app.id - environment: environment - } -} -``` - -Generate the app graph: - -```bash -rad app graph app.bicep -``` - -This creates `.radius/app-graph.json` with the full topology. - -## 2. View the Output - -The JSON output shows resources and their connections: - -```json -{ - "metadata": { - "generatedAt": "2026-02-04T00:58:00Z", - "radiusCliVersion": "0.35.0", - "sourceFiles": ["app.bicep"], - "sourceHash": "sha256:abc123...", - "gitCommit": "def456" - }, - "resources": [ - { - "id": ".../Applications.Core/containers/frontend", - "name": "frontend", - "type": "Applications.Core/containers", - "gitInfo": { - "commitSha": "def456...", - "author": "you@example.com", - "date": "2026-02-03T15:30:00Z", - "message": "Add frontend container" - } - } - // ... more resources - ], - "connections": [ - { - "sourceId": ".../containers/frontend", - "targetId": ".../containers/backend", - "type": "connection" - } - // ... more connections - ] -} -``` - -## 3. Generate Markdown Preview - -For a human-readable preview with Mermaid diagram: - -```bash -rad app graph app.bicep --format markdown -``` - -This creates both `.radius/app-graph.json` and `.radius/app-graph.md`: - -```markdown -# App Graph: myapp - -## Resources - -| Name | Type | Source | Last Commit | -|------|------|--------|-------------| -| frontend | Applications.Core/containers | app.bicep:12 | [def456](../../commit/def456) | -| backend | Applications.Core/containers | app.bicep:25 | [def456](../../commit/def456) | -| cache | Applications.Datastores/redisCaches | app.bicep:40 | [abc123](../../commit/abc123) | - -## Topology - -\```mermaid -graph LR - frontend[frontend] - backend[backend] - cache[(cache)] - - frontend --> backend - backend --> cache -\``` -``` - -## 4. Commit the Graph - -The app graph is designed to be committed alongside your Bicep files: - -```bash -git add app.bicep .radius/app-graph.json -git commit -m "Add Redis cache to application" -``` - -## 5. Set Up GitHub Action for PR Diffs - -Add `.github/workflows/app-graph-diff.yml`: - -```yaml -name: App Graph Diff - -on: - pull_request: - paths: - - '**/.radius/app-graph.json' - push: - branches: - - main - paths: - - '**/.radius/app-graph.json' - -permissions: - pull-requests: write - -jobs: - diff: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - fetch-depth: 0 - - - uses: radius-project/app-graph-diff-action@v1 - with: - github-token: ${{ secrets.GITHUB_TOKEN }} -``` - -When a PR changes the app graph, the Action posts a comment showing: -- Added/removed/modified resources -- New/removed connections -- Before/after Mermaid diagrams - -## 6. Validate Graph Freshness in CI - -Add validation to catch stale graphs: - -```yaml -name: Validate App Graph - -on: - pull_request: - paths: - - '**/*.bicep' - - '**/.radius/app-graph.json' - -jobs: - validate: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - name: Install Radius CLI - run: | - curl -fsSL https://get.radapp.io/install.sh | bash - rad bicep download - - - name: Validate graph is current - run: | - rad app graph app.bicep --stdout > /tmp/expected.json - diff .radius/app-graph.json /tmp/expected.json || { - echo "::error::App graph is stale. Run 'rad app graph app.bicep' and commit." - exit 1 - } -``` - -## Common Options - -| Option | Description | -|--------|-------------| -| `--stdout` | Write JSON to stdout instead of file | -| `-o ` | Write to custom output path | -| `--format markdown` | Also generate Markdown preview | -| `--no-git` | Skip git metadata (faster) | -| `--parameters ` | Use parameter file for Bicep | -| `--at ` | Generate graph at specific commit | - -## Troubleshooting - -### "Bicep CLI not found" - -```bash -rad bicep download -``` - -### "Not a git repository" - -Git metadata is optional. The graph generates successfully, but `gitInfo` fields show "not available". - -### "Stale graph detected in CI" - -Regenerate and commit: - -```bash -rad app graph app.bicep -git add .radius/app-graph.json -git commit --amend --no-edit -git push --force-with-lease -``` - -## Next Steps - -- [View graph history](./history.md) - Track architecture evolution over time -- [Compare environments](./environments.md) - See how portable types resolve differently -- [Customize the GitHub Action](./github-action.md) - Advanced configuration options diff --git a/specs/003-git-app-graph-storage/research.md b/specs/003-git-app-graph-storage/research.md deleted file mode 100644 index b25cdc6087..0000000000 --- a/specs/003-git-app-graph-storage/research.md +++ /dev/null @@ -1,239 +0,0 @@ -# Research: Git App Graph Preview - -**Feature Branch**: `001-git-app-graph-preview` -**Date**: February 4, 2026 - -## Research Tasks - -### 1. Radius CLI Architecture - -**Decision**: Extend existing `rad app graph` command in `pkg/cli/cmd/app/graph/` - -**Rationale**: -- The existing `rad app graph ` command already follows established CLI patterns -- Command structure uses Cobra framework with Runner pattern (`NewCommand()` + `Runner.Run()`) -- Output formatting is handled through `pkg/cli/output/` utilities -- Adding file-based input preserves conceptual consistency ("both are app graphs") - -**Alternatives Considered**: -- New `rad graph` top-level command: Rejected (breaks CLI hierarchy, less discoverable) -- New `rad bicep graph` command: Rejected (conceptually these are both "app graphs", just from different sources) - -### 2. Existing App Graph Data Structures - -**Decision**: Extend `ApplicationGraphResponse` and related types in `pkg/corerp/api/` - -**Rationale**: -- Existing types (`ApplicationGraphResource`, `ApplicationGraphConnection`) capture most needed fields -- Adding git metadata fields is additive and backward-compatible -- Deterministic output requires adding `sourceHash`, `sourceFile`, `sourceLine` metadata -- Types are already JSON-serializable via auto-generated marshalling code - -**Existing Structure** (from `zz_generated_models.go`): -```go -type ApplicationGraphResponse struct { - Resources []*ApplicationGraphResource -} - -type ApplicationGraphResource struct { - ID *string - Name *string - Type *string - ProvisioningState *string - Connections []*ApplicationGraphConnection - OutputResources []*ApplicationGraphOutputResource -} -``` - -**Additions Needed**: -- `GitInfo` struct: commit SHA, author, date, message -- `SourceFile`, `SourceLine` for Bicep source tracking -- `Metadata` struct: generatedAt, sourceFiles, sourceHash, radiusCliVersion - -### 3. Bicep Parsing Approach - -**Decision**: Use official Bicep CLI for compilation, then parse ARM JSON - -**Rationale** (per Constitution Principle VII - Simplicity Over Cleverness): -- Bicep CLI provides `bicep build --stdout` to compile to ARM JSON -- Radius already has Bicep CLI integration in `pkg/cli/bicep/` -- ARM JSON is stable and well-documented; parsing it avoids Bicep grammar complexity -- External modules are resolved by Bicep CLI, not our code - -**Alternatives Considered**: -- Custom Bicep parser: Rejected (complex, maintenance burden, grammar changes) -- ANTLR-based parser: Rejected (over-engineering for this use case) -- Use Bicep language server: Rejected (heavyweight, overkill for static analysis) - -**Implementation Pattern**: -```go -// Existing pattern in pkg/cli/bicep/types.go -func (impl *Impl) PrepareTemplate(filePath string) (map[string]any, error) { - args := []string{"build", "--stdout", filePath} - // Execute bicep CLI and parse JSON output -} -``` - -### 4. Graph Extraction from ARM JSON - -**Decision**: Extract resources and connections from compiled ARM JSON template - -**Rationale**: -- ARM JSON has stable schema with `resources` array -- Radius resources use `connections` and `routes` properties for relationships -- Existing graph logic in `pkg/corerp/frontend/controller/applications/graph_util.go` shows patterns - -**Key Extraction Points**: -1. `resources[].type` - Resource type identification -2. `resources[].name` - Resource name (may contain expressions) -3. `resources[].properties.connections` - Direct connections -4. `resources[].properties.routes` - Gateway routes -5. `resources[].dependsOn` - Explicit dependencies - -### 5. Git Integration - -**Decision**: Use `git log` and `git blame` via exec, not library - -**Rationale**: -- Git is universally available in development environments -- Shell commands are simpler than CGo bindings to libgit2 -- Radius already uses exec patterns for Bicep CLI -- Graceful degradation when not in git repo or shallow clone - -**Commands Needed**: -- `git blame -l -e ` - Get commit SHA per line -- `git log -1 --format='%H|%ae|%aI|%s' ` - Get commit metadata -- `git rev-parse --show-toplevel` - Detect git repo root - -### 6. GitHub Action Architecture - -**Decision**: Lightweight Action that reads committed JSON from git history; no graph generation - -**Rationale** (per spec Committed Artifact Model): -- Action only needs git and jq, not Bicep/Radius tooling -- Works in forks without special secrets -- Fast execution (JSON comparison vs. full compilation) -- Reproducible (graph captured at commit time) - -**Implementation**: -1. Checkout base and head commits -2. Read `.radius/app-graph.json` from each -3. Compute diff using JSON comparison -4. Render diff as Markdown with Mermaid diagrams -5. Post/update PR comment using `peter-evans/create-or-update-comment` - -**Alternatives Considered**: -- GitHub App: Rejected for MVP (more complex setup, centralized management) -- Generate graph in CI: Rejected (requires Bicep tooling, slower, less reproducible) - -### 7. Diff Computation Strategy - -**Decision**: JSON-based diffing with semantic resource comparison - -**Rationale**: -- JSON is deterministic when keys are sorted -- Resource ID provides stable identity across commits -- Diff should show added/removed/modified at resource level, not line level - -**Diff Algorithm**: -1. Parse base and head JSON -2. Create resource map keyed by ID -3. Compare: - - Added: ID in head but not base - - Removed: ID in base but not head - - Modified: ID in both but properties differ -4. For connections: compare by (sourceID, targetID) tuples - -### 8. Output Formats - -**Decision**: JSON canonical, Markdown additive with embedded Mermaid - -**Rationale** (per spec): -- JSON is machine-readable, deterministic, diffable -- Markdown renders in GitHub UI without additional tooling -- Mermaid diagrams supported natively by GitHub -- Separation allows different consumers (CI vs. humans) - -**JSON Schema**: -```json -{ - "metadata": { - "generatedAt": "2026-01-30T10:15:00Z", - "sourceFiles": ["app.bicep", "modules/database.bicep"], - "sourceHash": "sha256:abc123...", - "radiusCliVersion": "0.35.0" - }, - "resources": [...], - "connections": [...] -} -``` - -**Mermaid Shapes** (per spec): -- Containers: rectangles (`[name]`) -- Gateways: diamonds (`{name}`) -- Databases: cylinders (`[(name)]`) - -### 9. Platform Abstraction for Future Integrations - -**Decision**: Separate diff computation from rendering; platform-specific rendering layer - -**Rationale** (per user input: "design choices today won't limit us in the future"): -- Core diff logic (JSON comparison) is platform-agnostic -- GitHub-specific code isolated to: - - GitHub Action (workflow YAML) - - Markdown/Mermaid rendering - - PR comment posting -- Future GitLab integration would only need new rendering layer - -**Architecture**: -``` -┌─────────────────────────────────────────────────────────────────┐ -│ Platform-Agnostic │ -├─────────────────────────────────────────────────────────────────┤ -│ CLI (rad app graph) │ JSON Schema │ Diff Computation │ -│ Git Integration │ Data Model │ Core Rendering (MD) │ -└─────────────────────────────────────────────────────────────────┘ - │ - ┌─────────┴─────────┐ - ▼ ▼ - ┌───────────────┐ ┌───────────────┐ - │ GitHub Action │ │ GitLab CI │ - │ PR Comments │ │ MR Notes │ - │ Mermaid │ │ (Future) │ - └───────────────┘ └───────────────┘ -``` - -### 10. Radius Bicep Extension Compatibility - -**Decision**: Support Radius Bicep extension type definitions - -**Rationale**: -- Radius extends Bicep with custom types (`Applications.Core/containers`, etc.) -- These types must be recognized in ARM JSON output -- Type registry already exists in `pkg/corerp/api/` - -**Implementation**: -- Resource type detection checks for `Applications.*` prefix -- Portable types (`Radius.Data/store`) shown as-is in static graph -- Environment-resolved graph (P3) requires live Radius environment connection - -## Technology Decisions Summary - -| Area | Decision | Rationale | -|------|----------|-----------| -| CLI Framework | Cobra (existing) | Consistency with `rad` CLI | -| Bicep Parsing | Bicep CLI → ARM JSON | Simplicity, official support | -| Git Operations | Shell exec | Universal, no CGo | -| Data Structures | Extend existing types | Backward compatible | -| GitHub Integration | Action, not App | Simpler setup, fork-friendly | -| Diff Algorithm | JSON semantic diff | Deterministic, meaningful | -| Output | JSON + optional Markdown | Machine + human readable | -| Platform Abstraction | Rendering layer separation | Future GitLab support | - -## Open Questions Resolution - -1. **Bicep Compiler Integration**: ✅ Use official Bicep CLI (per Constitution Principle VII) -2. **Graph Storage**: ✅ Committed to `.radius/app-graph.json` (per spec) -3. **GitHub App vs Action**: ✅ GitHub Action for MVP (simpler, fork-friendly) -4. **Diff Visualization**: Table + Mermaid in PR comments -5. **Parameter Handling**: ✅ Require params file; fail with clear error listing missing required parameters if Bicep has required parameters (no defaults) but no `--parameters` flag provided diff --git a/specs/003-git-app-graph-storage/spec.md b/specs/003-git-app-graph-storage/spec.md deleted file mode 100644 index 28197dd929..0000000000 --- a/specs/003-git-app-graph-storage/spec.md +++ /dev/null @@ -1,430 +0,0 @@ -# Feature Specification: Git App Graph Preview - -**Feature Branch**: `001-git-app-graph-preview` -**Created**: January 30, 2026 -**Status**: Draft -**Input**: User description: "Radius currently stores the state of application deployments as an app graph within its data store. Today, the app graph does not get generated until the application is deployed. Help me build an app graph representation for applications that are defined (e.g. in an app.bicep file) but not yet deployed. Additionally, enrich the app graph representation with git changelog info (i.e. git commit data) so that I may use this data to visualize how the app graph changes over time (i.e. across commits). The ultimate goal is to be able to visualize the app graph and do diffs of the app graph in GitHub on PRs, commit comparisons, etc." - -## Clarifications - -### Session 2026-02-04 - -- Q: GitHub App vs Action for PR integration? → A: GitHub Action (fork-friendly, no installation approval required, aligns with existing Radius workflow patterns) -- Q: Diff visualization format in PR comments? → A: Table + Mermaid diagrams (change table for details, before/after diagrams for visual topology) -- Q: How to handle Bicep parameters without params file? → A: Require params file (fail with error if Bicep has required parameters but no `--parameters` provided) -- Q: GitHub Action trigger events? → A: `pull_request` + `push` (PR for review comments, push to main for baseline tracking) -- Q: Monorepo support with multiple app graphs? → A: Auto-detect all `**/.radius/app-graph.json` files; each diffed independently - -## Problem Statement - -Radius currently generates application graphs only after deployment, which means: -1. Developers cannot preview the app graph structure before deploying -2. There's no way to track how the application architecture evolves over time -3. PR reviewers cannot see the impact of Bicep changes on the overall application topology -4. No mechanism exists to compare app graphs across commits or branches - -This feature introduces **static app graph generation** from Bicep files and **git-aware graph versioning** to enable visualization and diffing in GitHub workflows. - -## User Scenarios & Testing *(mandatory)* - -### User Story 1 - Generate App Graph from Bicep Files (Priority: P1) - -As a **developer**, I want to generate an app graph from my `app.bicep` file without deploying, so I can preview the application topology and validate my changes locally. - -As a **platform engineer**, I want to review app graph changes in PRs, so I can ensure architectural changes align with organizational standards before deployment. - -**Why this priority**: This is the foundational capability. Without static graph generation, no other features can function. It delivers immediate value by enabling local validation. - -**Independent Test**: Can be fully tested by running a CLI command against a Bicep file and verifying the graph output matches expected structure. - -**Acceptance Scenarios**: - -1. **Given** a valid `app.bicep` file with container, gateway, and database resources, **When** I run `rad app graph app.bicep`, **Then** I receive a JSON graph representation showing all resources and their connections. - -2. **Given** a Bicep file with syntax errors, **When** I run `rad app graph app.bicep`, **Then** I receive a clear error message indicating the parsing failure with line/column information. - -3. **Given** a Bicep file referencing external modules, **When** I run `rad app graph app.bicep`, **Then** the graph includes resources from all referenced modules with proper dependency tracking. - -4. **Given** a Bicep file with parameterized values, **When** I run `rad app graph app.bicep --parameters params.json`, **Then** the graph reflects the resolved parameter values. - -5. **Given** a Bicep file with required parameters (no defaults) and no `--parameters` flag, **When** I run `rad app graph app.bicep`, **Then** I receive a clear error listing the missing required parameters. - -6. **Given** a Bicep file using the Radius Bicep extension types, **When** I run `rad app graph app.bicep`, **Then** the graph correctly identifies Radius-specific resource types and their relationships. - ---- - -### User Story 2 - Export Graph as Diff-Friendly Format (Priority: P1) - -As a developer, I want the app graph exported in a deterministic, diff-friendly format, so I can commit it to version control and see meaningful diffs when it changes. - -**Why this priority**: Critical for enabling GitHub integration. Without a stable, diffable format, PR visualization is impossible. - -**Independent Test**: Generate graph twice from identical Bicep, verify outputs are byte-identical. Modify Bicep, regenerate, verify diff shows only the changed elements. - -**Output Model**: JSON is the canonical data format, always generated. Markdown is a rendered preview of the JSON data, generated additively when requested. - -**Acceptance Scenarios**: - -1. **Given** an app graph, **When** I export it, **Then** the JSON output is deterministically sorted (alphabetical by resource ID) producing identical output for identical inputs. - -2. **Given** an app graph, **When** I run `rad app graph app.bicep`, **Then** JSON is written to `.radius/app-graph.json` (default location) serving as the single source of truth for all automation and diff operations. - -3. **Given** an app graph, **When** I run `rad app graph app.bicep --stdout`, **Then** JSON is written to stdout instead of a file. - -4. **Given** an app graph, **When** I run `rad app graph app.bicep --format markdown`, **Then** I receive **both** `.radius/app-graph.json` and `.radius/app-graph.md` containing: - - A resource table with name, type, source file, and git metadata - - An embedded Mermaid diagram showing the topology that GitHub renders automatically - -5. **Given** a graph exported to markdown, **When** viewed in GitHub, **Then** the Mermaid diagram renders as a visual flowchart with distinct shapes for resource types (containers as rectangles, gateways as diamonds, databases as cylinders). - -6. **Given** two app graphs from different commits, **When** I diff them, **Then** the diff is computed from JSON (not Markdown), and added/removed/modified resources are clearly identified. - ---- - -### User Story 3 - Git Metadata Enrichment (Priority: P2) - -As a developer, I want the app graph to automatically include git commit information, so I can track when and why each resource was added or modified. - -**Why this priority**: Builds on P1 capabilities to enable historical tracking. Valuable but not blocking core functionality. - -**Independent Test**: Generate graph from a Bicep file in a git repository, verify each resource includes commit SHA, author, and timestamp of last modification by default. - -**Acceptance Scenarios**: - -1. **Given** a Bicep file in a git repository, **When** I run `rad app graph app.bicep`, **Then** each resource automatically includes the commit SHA, author, date, and message of its last modification. - -2. **Given** a resource defined across multiple Bicep files, **When** I generate the graph, **Then** the resource shows the most recent commit that affected any of its defining files. - -3. **Given** a newly added resource not yet committed, **When** I generate the graph, **Then** the resource is marked as "uncommitted" with the current working directory state. - -4. **Given** a graph with git metadata, **When** I export to markdown, **Then** each resource row includes a linked commit SHA (e.g., `[abc123](../../commit/abc123)`). - -5. **Given** a Bicep file in a git repository, **When** I run `rad app graph app.bicep --no-git`, **Then** the graph is generated without git metadata for faster execution. - -6. **Given** a Bicep file outside a git repository, **When** I run `rad app graph app.bicep`, **Then** the graph is generated successfully with git fields marked as "not available". - ---- - -### User Story 4 - GitHub Action for PR Graph Diff (Priority: P2) - -As a PR reviewer, I want to see a visual diff of the app graph in PR comments, so I can understand the architectural impact of code changes without deploying. - -**Why this priority**: High-value GitHub integration, but depends on P1 capabilities being stable. - -**Operational Model**: The GitHub Action reads committed `.radius/app-graph.json` files from git history — it does NOT generate graphs on-demand. This keeps the Action lightweight (no Bicep/Radius tooling required) and fast. - -**Trigger Events**: The Action supports two trigger modes: -- **`pull_request`**: Posts diff comments on PRs when `.radius/app-graph.json` changes -- **`push` to main/default branch**: Updates baseline tracking for historical comparison - -**Monorepo Support**: The Action auto-detects all `**/.radius/app-graph.json` files in the repository. Each graph is diffed independently, with separate PR comment sections per application. - -**Independent Test**: Create a PR with Bicep changes and updated graph JSON, verify the action posts a comment showing before/after graph comparison. - -**Acceptance Scenarios**: - -1. **Given** a PR that includes changes to `.radius/app-graph.json`, **When** the GitHub Action runs, **Then** it reads the JSON from base and head commits and posts a comment showing the graph diff with added/removed/modified resources highlighted. - -2. **Given** a PR with no changes to `.radius/app-graph.json`, **When** the GitHub Action runs, **Then** it posts a comment indicating "No app graph changes detected." - -3. **Given** a PR that adds a new connection between resources, **When** the GitHub Action runs, **Then** the diff clearly shows the new connection with source and target resources. - -4. **Given** a PR comment already exists from a previous run, **When** the PR is updated and the action runs again, **Then** the existing comment is updated rather than creating a duplicate. - -5. **Given** a PR where Bicep files changed but `.radius/app-graph.json` was not updated, **When** the CI validation job runs, **Then** it fails with a message instructing the developer to run `rad app graph app.bicep` and commit the updated graph. - -6. **Given** a monorepo with multiple Radius applications (e.g., `apps/frontend/.radius/app-graph.json` and `apps/backend/.radius/app-graph.json`), **When** the GitHub Action runs on a PR, **Then** it detects all graph files and posts a unified comment with separate diff sections per application. - ---- - -### User Story 5 - Historical Graph Timeline (Priority: P3) - -As a developer, I want to view how my app graph evolved across commits, so I can understand architectural decisions and identify when changes were introduced. - -**Why this priority**: Advanced feature for historical analysis. Valuable for debugging and auditing but not essential for core workflow. - -**Independent Test**: Generate timeline for last 10 commits, verify each entry shows the graph state and changes from previous commit. - -**Acceptance Scenarios**: - -1. **Given** a git repository with multiple commits affecting Bicep files, **When** I run `rad app graph history app.bicep --commits 10`, **Then** I receive a timeline showing graph snapshots at each commit with change summaries. - -2. **Given** a specific commit SHA, **When** I run `rad app graph app.bicep --at abc123`, **Then** I receive the app graph as it existed at that commit. - -3. **Given** two commit SHAs, **When** I run `rad app graph diff app.bicep --from abc123 --to def456`, **Then** I receive a detailed diff showing all graph changes between those commits. - ---- - -### User Story 6 - Environment-Resolved Graph (Priority: P3) - -As a platform engineer, I want to see how abstract Radius types resolve to concrete infrastructure in a specific environment, so I can understand the actual resources that will be deployed. - -**Why this priority**: Advanced feature for environment-specific analysis. The static graph (showing portable types) serves most PR review needs; resolved views are valuable for deployment planning and troubleshooting. - -**Background**: Radius portable types like `Radius.Data/store` resolve differently depending on the environment's recipe configuration: -- Environment → RecipePack → Recipe → Concrete Resource -- The same `Radius.Data/store` might become PostgreSQL in `dev` and CosmosDB in `prod` - -**Independent Test**: Generate resolved graph for an environment with known recipe bindings, verify concrete resource types appear instead of abstract Radius types. - -**Acceptance Scenarios**: - -1. **Given** a Bicep file with `Radius.Data/store` and a connected Radius environment with PostgreSQL recipes, **When** I run `rad app graph app.bicep --environment prod`, **Then** the graph shows the resolved `Azure.DBforPostgreSQL/flexibleServers` (or equivalent) instead of the abstract `Radius.Data/store`. - -2. **Given** a Bicep file with portable types, **When** I run `rad app graph app.bicep --environment dev` and `rad app graph app.bicep --environment prod`, **Then** I can compare how the same application resolves to different infrastructure across environments. - -3. **Given** an environment where a recipe is not configured for a portable type, **When** I run `rad app graph app.bicep --environment prod`, **Then** the graph shows the abstract type with an annotation indicating "no recipe bound". - -4. **Given** a Bicep file, **When** I run `rad app graph app.bicep` (no `--environment` flag), **Then** the graph shows the abstract portable types (default behavior unchanged). - ---- - -### Edge Cases - -- What happens when Bicep file references resources outside the current file/module that cannot be resolved? - - Generate partial graph with unresolved references marked as "external" placeholders -- How does the system handle circular dependencies in Bicep? - - Detect and report cycles with clear error messaging; still generate graph with cycle annotation -- What happens when git history is shallow (e.g., `--depth 1` clone)? - - Gracefully degrade: use available history, mark resources as "history unavailable" when git blame fails -- How does the system handle large graphs (100+ resources)? - - Paginate CLI output, provide `--filter` options, optimize JSON/Markdown output for size -- What happens when Bicep uses runtime expressions that can't be statically resolved? - - Mark affected values as "dynamic" in the graph, use placeholder notation -- What happens when Bicep files use cloud-specific resources (Azure, AWS)? - - Graph generation MUST work regardless of cloud provider; cloud-specific resources are represented with their provider prefix (e.g., `Microsoft.Storage/storageAccounts`, `AWS::S3::Bucket`) -- What happens when the committed graph is stale (Bicep changed but graph not regenerated)? - - CI validation job compares committed graph to freshly generated graph; fails PR if they differ - - Graph JSON includes `sourceHash` to detect staleness without full regeneration - - Clear error message instructs developer to run `rad app graph app.bicep` -- What does the graph show for portable Radius types like `Radius.Data/store` that resolve differently per environment? - - **Static graph shows abstract types**: The declared `Radius.Data/store` is shown, not the resolved infrastructure (PostgreSQL, CosmosDB, etc.) - - This is intentional—the static graph represents the **portable application architecture** independent of environment-specific recipe resolution - - For environment-resolved views, see User Story 6 (P3) - ---- - -## CLI Design - -This feature extends the existing `rad app graph` command with file-based input for static graph generation. The command intelligently distinguishes between deployed apps and Bicep files based on the argument: - -| Command | Input Type | Output | -|---------|------------|--------| -| `rad app graph myapp` | App name | Deployed app graph (existing behavior) | -| `rad app graph myapp -e prod` | App name + environment | Deployed graph in specific environment | -| `rad app graph app.bicep` | Bicep file (`.bicep` extension) | JSON to `.radius/app-graph.json` (default) | -| `rad app graph app.bicep --stdout` | Bicep file + stdout flag | JSON to stdout (no file written) | -| `rad app graph app.bicep -o custom.json` | Bicep file + custom output | JSON to specified file | -| `rad app graph app.bicep --format markdown` | Bicep file + markdown | JSON + Markdown to `.radius/` | -| `rad app graph app.bicep --no-git` | Bicep file + no-git | JSON without git metadata (faster) | -| `rad app graph app.bicep --at abc123` | Bicep file + commit | JSON at specific commit | -| `rad app graph diff app.bicep --from abc123 --to def456` | Bicep file + commits | Diff computed from JSON, output as JSON or Markdown | -| `rad app graph history app.bicep --commits 10` | Bicep file + count | Historical timeline | -| `rad app graph app.bicep --environment prod` | Bicep file + environment | JSON with resolved recipe types | - -**Output Model**: -- **JSON is canonical**: Always generated, serves as the single source of truth for all automation and diff operations -- **Markdown is additive**: When `--format markdown` is specified, Markdown is generated *in addition to* JSON as a human-readable preview -- **GitHub Action uses JSON**: Diff computation is always JSON-to-JSON; Markdown is purely a rendering/presentation layer for PR comments - -**Design Rationale**: Unifying under `rad app graph` provides: -- Conceptual consistency: both are "app graphs" (prospective vs. deployed) -- Discoverability: all graph functionality in one place -- Intuitive disambiguation: `.bicep` extension clearly indicates file input -- Alignment with existing `rad app graph ` pattern - ---- - -## Committed Artifact Model - -The app graph JSON is designed to be **committed to version control** as a tracked artifact. This enables lightweight GitHub integration without requiring the Action to have Bicep/Radius tooling. - -### Default Output Location - -By default, `rad app graph app.bicep` writes to `.radius/app-graph.json` relative to the Bicep file's directory: - -``` -myapp/ -├── app.bicep -├── modules/ -│ └── database.bicep -└── .radius/ - ├── app-graph.json # Canonical graph data (committed) - └── app-graph.md # Optional preview (if --format markdown) -``` - -### Developer Workflow - -```bash -# 1. Make changes to Bicep files -vim app.bicep - -# 2. Regenerate the graph (writes to .radius/app-graph.json by default) -rad app graph app.bicep - -# 3. Commit both the Bicep changes and updated graph -git add app.bicep .radius/app-graph.json -git commit -m "Add redis cache to application" - -# 4. Push and create PR — GitHub Action reads committed JSON to render diff -git push -``` - -### Why Committed Artifacts? - -| Benefit | Explanation | -|---------|-------------| -| **Simple GitHub Action** | Action is a lightweight viewer that reads JSON from git history — no Bicep CLI, no Radius environment needed | -| **Fast CI** | No graph generation in CI; diff is just JSON comparison | -| **Reproducible** | Graph captured at commit time, not regenerated with potentially different tooling | -| **Auditable** | Graph evolution visible in git history alongside code changes | -| **Fork-friendly** | Works in forks without special tooling or secrets | - -### Staleness Detection - -To prevent committed graphs from drifting out of sync with Bicep files: - -1. **CI Validation Job** (recommended): Regenerate graph in CI, compare to committed version, fail if different -2. **Pre-commit Hook** (optional): Validate graph matches Bicep before allowing commit -3. **Graph Metadata**: JSON includes `sourceHash` field — hash of input Bicep file(s) for staleness detection - -```json -{ - "metadata": { - "generatedAt": "2026-01-30T10:15:00Z", - "sourceFiles": ["app.bicep", "modules/database.bicep"], - "sourceHash": "sha256:abc123...", - "radiusCliVersion": "0.35.0" - }, - "resources": [...], - "connections": [...] -} -``` - ---- - -## Requirements *(mandatory)* - -### Functional Requirements - -- **FR-001**: System MUST parse Bicep files and extract resource definitions without requiring deployment -- **FR-002**: System MUST resolve module references and build a complete graph across multiple Bicep files -- **FR-003**: System MUST extract connection relationships from resource properties (connections, routes, ports) -- **FR-004**: System MUST produce deterministic output (same input = byte-identical output) -- **FR-005**: System MUST always generate JSON output as the canonical data format with stable key ordering for deterministic diffs -- **FR-006**: System MUST support Markdown output as an **additive** preview format (generated alongside JSON when `--format markdown` is specified), containing a resource table and embedded Mermaid diagram -- **FR-007**: System MUST perform all diff computations using JSON data, with Markdown used only as a rendering layer for human consumption -- **FR-008**: System MUST enrich graph nodes with git metadata (commit SHA, author, date, message) by default when in a git repository, with `--no-git` flag to disable -- **FR-009**: System MUST track which Bicep file(s) define each resource for git blame integration -- **FR-010**: System MUST write graph output to `.radius/app-graph.json` by default (relative to Bicep file location), with `--stdout` flag for stdout output and `-o` flag for custom path -- **FR-011**: System MUST include `sourceHash` metadata in JSON output to enable staleness detection -- **FR-012**: System MUST provide a GitHub Action that reads committed graph JSON from git history and posts graph diffs on PRs (no graph generation in CI) -- **FR-013**: System MUST update existing PR comments rather than creating duplicates -- **FR-014**: System MUST support generating graphs at specific git commits/refs -- **FR-015**: System MUST handle Bicep parameter files to resolve parameterized values -- **FR-016**: System MUST report clear errors for invalid Bicep syntax with file/line/column information -- **FR-017**: System MUST handle unresolvable references gracefully with placeholder annotations -- **FR-018**: System MUST work with Bicep files targeting any cloud provider (multi-cloud neutrality per Constitution Principle III) -- **FR-019**: System MUST be compatible with the Radius Bicep extension type definitions - -### Non-Functional Requirements - -- **NFR-001**: All Go code MUST follow Effective Go patterns and pass `golangci-lint` (Constitution Principle II) -- **NFR-002**: All exported Go packages, types, and functions MUST have godoc comments (Constitution Code Quality Standards) -- **NFR-003**: Feature MUST NOT require changes to existing deployment workflows (Constitution Principle IX - Incremental Adoption) -- **NFR-004**: CLI commands MUST follow existing `rad` CLI patterns and conventions -- **NFR-005**: Error messages MUST be actionable with clear guidance for resolution (Constitution Principle VI) - -### Key Entities - -- **AppGraph**: Root container holding all resources, connections, and metadata for a single application - - Resources: Collection of AppGraphResource nodes - - Metadata: Git commit info, generation timestamp, source files - -- **AppGraphResource**: Single resource node in the graph - - ID: Unique resource identifier (matches Radius resource ID format) - - Name: Human-readable resource name - - Type: Resource type (e.g., `Applications.Core/containers`) - - SourceFile: Bicep file path where resource is defined - - SourceLine: Line number in source file - - Connections: Outbound connections to other resources - - GitInfo: Last commit SHA, author, date, message affecting this resource - -- **AppGraphConnection**: Edge between two resources - - SourceID: Origin resource - - TargetID: Destination resource - - Direction: Outbound/Inbound - - Type: Connection type (connection, route, port binding) - -- **GraphDiff**: Comparison result between two graphs - - AddedResources: Resources present in new but not old - - RemovedResources: Resources present in old but not new - - ModifiedResources: Resources with changed properties/connections - - AddedConnections: New edges - - RemovedConnections: Removed edges - ---- - -## Success Criteria *(mandatory)* - -### Measurable Outcomes - -- **SC-001**: Graph generation completes in < 5 seconds for applications with up to 50 resources -- **SC-002**: Generated graphs are 100% deterministic (identical input produces byte-identical output) -- **SC-003**: Graph diff correctly identifies all added, removed, and modified resources with zero false positives -- **SC-004**: GitHub Action posts PR comments within 60 seconds of workflow trigger -- **SC-005**: Markdown output (including embedded Mermaid diagram) renders correctly in GitHub without manual formatting -- **SC-006**: Git enrichment adds < 2 seconds overhead for repositories with up to 1000 commits -- **SC-007**: System handles Bicep files up to 5000 lines without performance degradation -- **SC-008**: Error messages include actionable guidance in 100% of failure cases - ---- - -## Testing Requirements *(per Constitution Principle IV)* - -This feature MUST include comprehensive testing across the testing pyramid: - -### Unit Tests -- Test individual graph parsing functions in isolation -- Test git metadata extraction logic -- Test output formatters (JSON, Markdown with embedded Mermaid) with known inputs -- Test error handling for malformed Bicep files -- All unit tests runnable with `make test` without external dependencies - -### Integration Tests -- Test Bicep CLI integration for file parsing -- Test git operations (blame, log) with real git repositories -- Test module resolution across multiple Bicep files - -### Functional Tests -- End-to-end test: Bicep file → graph generation → output validation -- Test GitHub Action in a real PR workflow -- Test graph diff accuracy with known before/after states - ---- - -## Open Questions - -1. **Bicep Compiler Integration**: Should we use the official Bicep CLI for parsing, or implement a lightweight parser? Trade-off: accuracy vs. dependency management. **Recommendation**: Use official Bicep CLI per Constitution Principle VII (Simplicity Over Cleverness). - -2. ~~**Graph Storage**: Should generated graphs be committed to the repo (e.g., `app-graph.json`)? Trade-off: visibility vs. repo noise.~~ **RESOLVED (Initial Implementation)**: Graphs are committed to `.radius/app-graph.json`. This enables lightweight GitHub Action (reads from git history, no tooling required) and provides auditable graph evolution. **Future Evolution**: External storage backends (e.g., SQLite, cloud databases) could be supported for scenarios requiring graph queries across repositories, historical analytics, or enterprise-scale graph management. - -3. ~~**GitHub App vs Action**: Should the PR integration be a GitHub Action (user-managed) or a GitHub App (centrally managed)? Trade-off: flexibility vs. ease of setup.~~ **RESOLVED**: GitHub Action. Fork-friendly, no installation approval required, aligns with existing Radius workflow patterns, supports incremental adoption. - -4. ~~**Diff Visualization**: What's the preferred format for showing diffs in PR comments—table-based, Mermaid side-by-side, or unified text diff?~~ **RESOLVED**: Table + Mermaid diagrams. Change table shows added/removed/modified resources with details; before/after Mermaid diagrams provide visual topology comparison. Both render natively in GitHub. - -5. ~~**Parameter Handling**: How should we handle Bicep parameters without a params file—use defaults, require params, or mark as "unknown"?~~ **RESOLVED**: Require params file. If Bicep has required parameters (no defaults) but no `--parameters` flag is provided, fail with a clear error message listing the missing parameters. - ---- - -## Cross-Repository Impact *(per Constitution Principle XVII)* - -This feature may affect multiple Radius repositories: - -| Repository | Impact | -|------------|--------| -| `radius` | CLI implementation (`pkg/cli/cmd/app/graph/`), core graph logic | -| `docs` | User documentation for new CLI commands, GitHub Action setup guide | -| `design-notes` | This specification and implementation plan | - -Coordinate changes across repositories per Constitution guidance on polyglot project coherence. \ No newline at end of file diff --git a/specs/003-git-app-graph-storage/tasks.md b/specs/003-git-app-graph-storage/tasks.md deleted file mode 100644 index 0e946ed2f9..0000000000 --- a/specs/003-git-app-graph-storage/tasks.md +++ /dev/null @@ -1,323 +0,0 @@ -# Tasks: Git App Graph Preview - -**Input**: Design documents from `/specs/001-git-app-graph-preview/` -**Prerequisites**: plan.md ✓, spec.md ✓, research.md ✓, data-model.md ✓, contracts/ ✓, quickstart.md ✓ - -**Tests**: Included per spec.md "Testing Requirements" section - comprehensive testing across the testing pyramid. - -**Organization**: Tasks grouped by user story for independent implementation and testing. - -## Format: `[ID] [P?] [Story] Description` - -- **[P]**: Can run in parallel (different files, no dependencies) -- **[Story]**: Which user story this task belongs to (US1, US2, US3, US4, US5, US6) -- Exact file paths for radius repository - ---- - -## Phase 1: Setup (Shared Infrastructure) - -**Purpose**: Project initialization and scaffolding for the static graph feature - -- [ ] T001 Create package structure `pkg/cli/bicep/` with package doc in pkg/cli/bicep/doc.go -- [ ] T002 Create package structure `pkg/cli/git/` with package doc in pkg/cli/git/doc.go -- [ ] T003 [P] Add new test directories: test/unit/cli/graph/, test/integration/cli/graph/, test/functional/cli/graph/ -- [ ] T004 [P] Define AppGraph types in pkg/corerp/api/v20231001preview/appgraph_static_types.go -- [ ] T005 [P] Define GraphDiff types in pkg/corerp/api/v20231001preview/appgraph_diff_types.go -- [ ] T006 Add JSON schema validation helpers in pkg/cli/output/json.go (deterministic key ordering) - ---- - -## Phase 2: Foundational (Blocking Prerequisites) - -**Purpose**: Core infrastructure required by ALL user stories - -**⚠️ CRITICAL**: No user story work can begin until this phase is complete - -- [ ] T007 Implement Bicep CLI executor in pkg/cli/bicep/executor.go (wraps `bicep build --stdout`) -- [ ] T008 Implement ARM JSON parser interface in pkg/cli/bicep/parser.go (extract resources array) -- [ ] T009 Implement resource extraction from ARM JSON in pkg/cli/bicep/extractor.go -- [ ] T010 Implement connection detection from resource properties in pkg/cli/bicep/connections.go -- [ ] T011 [P] Implement source hash computation (SHA256 of input files) in pkg/cli/bicep/hash.go -- [ ] T012 [P] Add Bicep file detection logic (`.bicep` extension) in pkg/cli/cmd/app/graph/detect.go -- [ ] T013 Unit tests for Bicep executor in test/unit/cli/graph/executor_test.go -- [ ] T014 Unit tests for ARM JSON parser in test/unit/cli/graph/parser_test.go - -**Checkpoint**: Foundation ready - user story implementation can begin - ---- - -## Phase 3: User Story 1 - Generate App Graph from Bicep Files (Priority: P1) 🎯 MVP - -**Goal**: Enable `rad app graph app.bicep` to generate a JSON app graph from Bicep files without deployment - -**Independent Test**: Run CLI against sample Bicep file, verify JSON output contains expected resources and connections - -### Tests for User Story 1 - -- [ ] T015 [P] [US1] Unit test for graph generation with valid Bicep in test/unit/cli/graph/static_test.go -- [ ] T016 [P] [US1] Unit test for error handling with invalid Bicep in test/unit/cli/graph/static_errors_test.go -- [ ] T017 [P] [US1] Unit test for module resolution in test/unit/cli/graph/modules_test.go -- [ ] T018 [US1] Integration test with real Bicep CLI in test/integration/cli/graph/bicep_integration_test.go - -### Implementation for User Story 1 - -- [ ] T019 [US1] Implement static graph generator in pkg/cli/cmd/app/graph/static.go -- [ ] T020 [US1] Add file input detection to existing graph.go entry point in pkg/cli/cmd/app/graph/graph.go -- [ ] T021 [US1] Implement module resolution (transitive Bicep imports) in pkg/cli/bicep/modules.go -- [ ] T022 [US1] Add parameter file support (`--parameters`) in pkg/cli/cmd/app/graph/params.go -- [ ] T023 [US1] Implement required parameter validation (fail if missing) in pkg/cli/bicep/params.go -- [ ] T024 [US1] Add Radius Bicep extension type detection in pkg/cli/bicep/radius_types.go -- [ ] T025 [US1] Implement error handling with line/column info in pkg/cli/bicep/errors.go -- [ ] T026 [US1] Add logging for graph generation operations in pkg/cli/cmd/app/graph/static.go - -**Checkpoint**: `rad app graph app.bicep` generates valid JSON graph from Bicep files - ---- - -## Phase 4: User Story 2 - Export Graph as Diff-Friendly Format (Priority: P1) - -**Goal**: Output deterministic JSON to `.radius/app-graph.json` and optional Markdown with Mermaid diagrams - -**Independent Test**: Generate graph twice from identical input, verify byte-identical output; verify Markdown renders in GitHub - -### Tests for User Story 2 - -- [ ] T027 [P] [US2] Unit test for deterministic JSON output in test/unit/cli/graph/json_determinism_test.go -- [ ] T028 [P] [US2] Unit test for Mermaid diagram generation in test/unit/cli/graph/mermaid_test.go -- [ ] T029 [P] [US2] Unit test for Markdown table formatting in test/unit/cli/graph/markdown_test.go -- [ ] T030 [US2] Integration test for file output paths in test/integration/cli/graph/output_test.go - -### Implementation for User Story 2 - -- [ ] T031 [US2] Implement deterministic JSON serializer in pkg/cli/output/deterministic_json.go -- [ ] T032 [US2] Implement default output path (`.radius/app-graph.json`) in pkg/cli/cmd/app/graph/output.go -- [ ] T033 [US2] Add `--stdout` flag for stdout output in pkg/cli/cmd/app/graph/flags.go -- [ ] T034 [US2] Add `-o` flag for custom output path in pkg/cli/cmd/app/graph/flags.go -- [ ] T035 [US2] Implement Mermaid diagram generator in pkg/cli/output/mermaid.go -- [ ] T036 [US2] Implement Markdown table formatter in pkg/cli/output/markdown.go -- [ ] T037 [US2] Add `--format markdown` flag (generates both JSON + Markdown) in pkg/cli/cmd/app/graph/flags.go -- [ ] T038 [US2] Integrate display.go with new output formatters in pkg/cli/cmd/app/graph/display.go -- [ ] T039 [US2] Add Mermaid shape mapping (containers→rectangles, gateways→diamonds, databases→cylinders) in pkg/cli/output/mermaid.go - -**Checkpoint**: Deterministic JSON output + optional Markdown with Mermaid diagrams working - ---- - -## Phase 5: User Story 3 - Git Metadata Enrichment (Priority: P2) - -**Goal**: Automatically enrich each resource with git commit information (SHA, author, date, message) - -**Independent Test**: Generate graph in git repo, verify each resource has git metadata; verify `--no-git` skips enrichment - -### Tests for User Story 3 - -- [ ] T040 [P] [US3] Unit test for git blame parsing in test/unit/cli/graph/git_blame_test.go -- [ ] T041 [P] [US3] Unit test for git log parsing in test/unit/cli/graph/git_log_test.go -- [ ] T042 [P] [US3] Unit test for uncommitted file detection in test/unit/cli/graph/git_uncommitted_test.go -- [ ] T043 [US3] Integration test with real git repository in test/integration/cli/graph/git_integration_test.go - -### Implementation for User Story 3 - -- [ ] T044 [US3] Implement git repository detection in pkg/cli/git/repo.go -- [ ] T045 [US3] Implement git blame executor in pkg/cli/git/blame.go -- [ ] T046 [US3] Implement git log metadata extraction in pkg/cli/git/log.go -- [ ] T047 [US3] Implement per-resource git info enrichment in pkg/cli/git/metadata.go -- [ ] T048 [US3] Add uncommitted changes detection in pkg/cli/git/status.go -- [ ] T049 [US3] Add `--no-git` flag for faster execution in pkg/cli/cmd/app/graph/flags.go -- [ ] T050 [US3] Handle non-git directories gracefully in pkg/cli/git/metadata.go -- [ ] T051 [US3] Handle shallow clones (mark as "history unavailable") in pkg/cli/git/blame.go -- [ ] T052 [US3] Add linked commit SHA in Markdown output in pkg/cli/output/markdown.go - -**Checkpoint**: Graph resources include git metadata by default; `--no-git` works - ---- - -## Phase 6: User Story 4 - GitHub Action for PR Graph Diff (Priority: P2) - -**Goal**: GitHub Action reads committed JSON from git history and posts diff comments on PRs - -**Independent Test**: Create PR with graph changes, verify Action posts comment with change table and Mermaid diagrams - -### Tests for User Story 4 - -- [ ] T053 [P] [US4] Unit test for JSON-to-JSON diff computation in test/unit/cli/graph/diff_test.go -- [ ] T054 [P] [US4] Unit test for diff summary generation in test/unit/cli/graph/diff_summary_test.go -- [ ] T055 [US4] Unit test for diff Markdown rendering in test/unit/cli/graph/diff_render_test.go - -### Implementation for User Story 4 - -- [ ] T056 [US4] Implement graph diff computation in pkg/cli/cmd/app/graph/diff.go -- [ ] T057 [US4] Implement resource comparison (by ID) in pkg/cli/cmd/app/graph/diff.go -- [ ] T058 [US4] Implement connection comparison (by source+target tuple) in pkg/cli/cmd/app/graph/diff.go -- [ ] T059 [US4] Implement diff summary statistics in pkg/cli/cmd/app/graph/diff.go -- [ ] T060 [US4] Implement diff Markdown renderer (change table + before/after Mermaid) in pkg/cli/output/diff_markdown.go -- [ ] T061 [US4] Create GitHub Action definition in actions/app-graph-diff/action.yml -- [ ] T062 [US4] Implement diff computation shell script in actions/app-graph-diff/diff.sh -- [ ] T063 [US4] Implement Mermaid rendering for Action in actions/app-graph-diff/render.js -- [ ] T064 [US4] Add PR comment create-or-update logic in actions/app-graph-diff/comment.sh -- [ ] T065 [US4] Add monorepo support (glob `**/.radius/app-graph.json`) in actions/app-graph-diff/detect.sh -- [ ] T066 [US4] Add staleness validation (compare committed vs regenerated) in actions/app-graph-diff/validate.sh -- [ ] T067 [US4] Add `push` trigger support for baseline tracking in actions/app-graph-diff/action.yml - -**Checkpoint**: GitHub Action posts diff comments on PRs with graph changes - ---- - -## Phase 7: User Story 5 - Historical Graph Timeline (Priority: P3) - -**Goal**: Enable `rad app graph history` to show graph evolution across commits - -**Independent Test**: Run history command on repo with multiple commits, verify timeline shows graph changes - -### Tests for User Story 5 - -- [ ] T068 [P] [US5] Unit test for timeline generation in test/unit/cli/graph/history_test.go -- [ ] T069 [US5] Integration test with multi-commit git history in test/integration/cli/graph/history_integration_test.go - -### Implementation for User Story 5 - -- [ ] T070 [US5] Implement `rad app graph history` subcommand in pkg/cli/cmd/app/graph/history.go -- [ ] T071 [US5] Add `--commits N` flag in pkg/cli/cmd/app/graph/history.go -- [ ] T072 [US5] Implement `--at ` flag for graph at specific commit in pkg/cli/cmd/app/graph/flags.go -- [ ] T073 [US5] Implement `rad app graph diff --from X --to Y` subcommand in pkg/cli/cmd/app/graph/diff.go -- [ ] T074 [US5] Implement commit range iteration in pkg/cli/git/history.go -- [ ] T075 [US5] Implement change summary per commit in pkg/cli/cmd/app/graph/history.go - -**Checkpoint**: Historical timeline and commit-specific graph generation working - ---- - -## Phase 8: User Story 6 - Environment-Resolved Graph (Priority: P3) - -**Goal**: Enable `rad app graph --environment` to show resolved recipe types instead of abstract Radius types - -**Independent Test**: Generate graph with environment flag, verify abstract types resolve to concrete infrastructure types - -### Tests for User Story 6 - -- [ ] T076 [P] [US6] Unit test for recipe type resolution in test/unit/cli/graph/resolve_test.go -- [ ] T077 [US6] Integration test with Radius environment in test/integration/cli/graph/environment_test.go - -### Implementation for User Story 6 - -- [ ] T078 [US6] Add `--environment` flag in pkg/cli/cmd/app/graph/flags.go -- [ ] T079 [US6] Implement environment connection in pkg/cli/cmd/app/graph/environment.go -- [ ] T080 [US6] Implement recipe lookup from environment in pkg/cli/cmd/app/graph/recipes.go -- [ ] T081 [US6] Implement portable type → concrete type resolution in pkg/cli/cmd/app/graph/resolve.go -- [ ] T082 [US6] Add "no recipe bound" annotation for unbound types in pkg/cli/cmd/app/graph/resolve.go -- [ ] T083 [US6] Integrate resolved types into graph output in pkg/cli/cmd/app/graph/static.go - -**Checkpoint**: Environment-resolved graphs show concrete infrastructure types - ---- - -## Phase 9: Polish & Cross-Cutting Concerns - -**Purpose**: Documentation, validation, and refinements - -- [ ] T084 [P] Update pkg/cli/cmd/app/graph/README.md with new command documentation -- [ ] T085 [P] Add godoc comments to all exported types in pkg/cli/bicep/ -- [ ] T086 [P] Add godoc comments to all exported types in pkg/cli/git/ -- [ ] T087 Run golangci-lint and fix any issues in new code -- [ ] T088 [P] Create functional E2E test in test/functional/cli/graph/e2e_test.go -- [ ] T089 Validate quickstart.md scenarios work end-to-end -- [ ] T090 Update CHANGELOG.md with new feature description -- [ ] T091 [P] Create GitHub Action README in actions/app-graph-diff/README.md - ---- - -## Dependencies & Execution Order - -### Phase Dependencies - -- **Setup (Phase 1)**: No dependencies - can start immediately -- **Foundational (Phase 2)**: Depends on Setup - BLOCKS all user stories -- **US1 (Phase 3)**: Depends on Foundational - First MVP -- **US2 (Phase 4)**: Depends on Foundational - Second MVP capability -- **US3 (Phase 5)**: Depends on US1/US2 working - Enrichment layer -- **US4 (Phase 6)**: Depends on US2 (diff-friendly output) + US3 (git metadata) -- **US5 (Phase 7)**: Depends on US3 (git integration) -- **US6 (Phase 8)**: Can start after US1 - independent branch -- **Polish (Phase 9)**: After all desired user stories complete - -### User Story Dependencies - -``` -Foundational (Phase 2) - │ - ├──────────────┬──────────────┐ - ▼ ▼ ▼ - US1 (P1) US2 (P1) US6 (P3) - │ │ (independent) - └──────┬───────┘ - ▼ - US3 (P2) - │ - ┌──────┴──────┐ - ▼ ▼ - US4 (P2) US5 (P3) -``` - -### Parallel Opportunities - -**Phase 1 (Setup)**: T001, T002 sequential; T003-T006 all parallel -**Phase 2 (Foundational)**: T007-T010 sequential; T011-T012 parallel; T013-T014 parallel -**Each User Story**: All tests marked [P] can run in parallel; implementation tasks mostly sequential within story - ---- - -## Parallel Example: User Story 1 Tests - -```bash -# Launch all US1 tests in parallel: -Task: "T015 [P] [US1] Unit test for graph generation with valid Bicep" -Task: "T016 [P] [US1] Unit test for error handling with invalid Bicep" -Task: "T017 [P] [US1] Unit test for module resolution" - -# Then run integration test: -Task: "T018 [US1] Integration test with real Bicep CLI" -``` - ---- - -## Implementation Strategy - -### MVP First (US1 + US2 = P1 Stories) - -1. Complete Phase 1: Setup -2. Complete Phase 2: Foundational (CRITICAL) -3. Complete Phase 3: User Story 1 (core graph generation) -4. Complete Phase 4: User Story 2 (diff-friendly output) -5. **STOP and VALIDATE**: Run quickstart.md scenarios locally -6. Deploy CLI changes for developer feedback - -### Git Integration (P2 Stories) - -1. Add US3: Git metadata enrichment -2. Add US4: GitHub Action for PR diffs -3. **VALIDATE**: Test in real PR workflow - -### Advanced Features (P3 Stories) - -1. Add US5: Historical timeline (optional) -2. Add US6: Environment resolution (optional) -3. Final polish phase - -### Parallel Team Strategy - -With multiple developers: -- **Developer A**: Phase 1 → Phase 2 → US1 → US3 → US5 -- **Developer B**: Phase 1 (parallel) → US2 → US4 (GitHub Action) -- **Developer C**: US6 (can start after Phase 2) - ---- - -## Notes - -- Radius repository is at `../radius` relative to design-notes -- All Go code must pass `golangci-lint` (Constitution Principle II) -- All exported types need godoc comments (NFR-002) -- JSON output must be deterministic (FR-004, SC-002) -- Existing `rad app graph ` behavior must not change (FR-003) -- GitHub Action is lightweight (reads JSON only, no Bicep tooling)