Radius Resource Types and Recipes for Github-Radius integration

eng/design-notes/extensibility/2026-05-radius-resource-types-recipes.md

@@ -0,0 +1,215 @@
+# Radius Resource Types & Recipes
+
+* **Author**: Reshma Abdul Rahim (@Reshrahim)
+
+## Summary
+
+As part of GitHub-Radius integration, AI agents need to understand application source code and automatically generate deployment definitions (`app.bicep`). For the agents to work **deterministically and reliably**, they need a well-defined catalog of application-oriented resource types backed by production-ready recipes.
+
+Radius maintains the resource type schemas and recipes in the `resource-types-contrib` repository. We need about 30 application oriented resource types covering the basics: databases, caches, messaging, storage, and so on. These are what AI agents use to generate `app.bicep`, so the schemas need to be well-defined.
+
+Recipes reference community modules directly rather than custom IaC code. The `resource-types-contrib` repository contains type definitions and tested module references. For Azure, recipes point at Azure Verified Modules. For AWS, the Terraform Registry. For Kubernetes, Helm charts. Radius resolves inputs and outputs automatically, with the mapping configuration maintained in Recipe packs.
+
+In most cases, there is no IaC code to maintain — just a module reference and mapping configuration. In rare cases where no suitable community module exists or where Radius-specific orchestration is needed (e.g., the container recipe that manages Kubernetes deployments directly), a custom recipe module is still required. This document lays out the strategy to build and maintain the types and recipes for the GitHub-Radius integration to be successful.
+
+## Goals
+
+1. Build a resource type catalog broad enough for AI agents to generate accurate `app.bicep` for real-world applications.
+2. Minimize recipe authoring by pointing directly at community modules where possible. Custom recipe code is only needed when no suitable upstream module exists or Radius-specific orchestration is required.
+3. Establish a contribution model that lets the community add and validate resource types with clear maturity gates from Alpha to Stable.
+4. Extend recipe driver coverage to match where developers deploy: Bicep for Azure (via AVM), Terraform for AWS , Helm for Kubernetes.
+
+## 1. Resource Types
+
+Resource types are the building blocks of the application definition. Today's catalog is limited to a handful of types that serve only the Radius `todo-list` sample. To support real-world applications, the catalog needs to grow to cover the application dependencies developers actually use.
+
+A data-driven analysis by Copilot from cloud provider catalogs, Docker Hub, the Stack Overflow 2025 Developer Survey, IaC registries, and package registry trends identified 27 application components ranked by actual developer adoption. Adoption is measured by dedicated client-library downloads across four ecosystems (npm, PyPI, NuGet, RubyGems), weighted by survey usage, Docker pulls, and cloud availability. The full ranked catalog with methodology is in [`resource-type-ranked-catalog`](2026-05-radius-resource-types-recipes/resource-type-ranked-catalog.md).
+
+The top 27 break into three tiers:
+
+| Tier | What's included | Criteria |
+|------|----------------|----------|
+| **Build First** | PostgreSQL, Redis, Object Storage, OpenAI-compatible API, MongoDB, MySQL, Kafka, Elasticsearch/OpenSearch, RabbitMQ, SQL Server | Highest adoption + stable connection contracts suitable for cross-cloud abstraction |
+| **Build Next** | Serverless Functions, Message Queue (SQS/Azure Queue/Service Bus Queues), MQTT, pgvector, NATS, Oracle, Neo4j, Vault, Cassandra, InfluxDB | Strong adoption but higher abstraction complexity or narrower use cases |
+| **Build Later** | Ollama, Pub/Sub, ClickHouse, Keycloak, Spark, MLflow, Memcached | Emerging, niche, or platform-specific — build as demand materializes |
+
+Notable inclusions: OpenAI-compatible API reflects AI becoming a first-class application dependency (81.4% of surveyed developers use OpenAI GPT models). The resource type represents the chat/completions API contract implemented by OpenAI, Azure OpenAI, Anthropic, and compatible providers (Ollama, vLLM). pgvector (#14) is the recommended vector-database entry point with the same PostgreSQL connection contract and 3/3 cloud availability. Vault (#18) is included because applications directly establish runtime connections to secrets providers, unlike org-level identity or observability platforms.
+
+Shared infrastructure services (identity/auth, observability, logging, email, feature flags) are provisioned at the platform level, but applications still connect to them at runtime. For these, the environment provides connection metadata (endpoint, credentials) without a recipe — no infrastructure is provisioned per-application.
+
+## 2. Recipes
+
+Recipes are how Radius deploys infrastructure behind resource types. Though it is a concept of Radius, the implementation uses existing IaC languages, Bicep and Terraform. Instead of us writing the IaC code, we leverage well established community modules directly. This approach helps Radius to not maintain recipe code reducing the supply chain surface. There is no wrapper code to audit, patch, or keep in sync with upstream module changes.
+
+### Community Module Ecosystems
+
+Each cloud platform has a dominant IaC tool with an established library of reusable modules. These libraries are what Radius recipes reference:
+
+| Cloud / Platform | IaC Tool | Module Library | Registry |
+|------------------|----------|----------------|----------|
+| Azure | Bicep | [Azure Verified Modules (AVM)](https://aka.ms/avm) | `mcr.microsoft.com/bicep/avm/` |
+| AWS | Terraform | [terraform-aws-modules](https://registry.terraform.io/namespaces/terraform-aws-modules) | `registry.terraform.io/terraform-aws-modules/` |
+| Kubernetes | Helm | [Bitnami Charts](https://github.com/bitnami/charts) | `oci://registry-1.docker.io/bitnamicharts/` |
+
+> **Why not CloudFormation for AWS?** Based on my analysis across module sources, Terraform is preferred by Multi-cloud companies and [Terraform's registry](https://registry.terraform.io/namespaces/terraform-aws-modules) had wide usage with million weekly downloads across multiple modules. AWS CDK is preferred by AWS native teams, and internally it uses CloudFormation but no clear authentic source of CloudFormation usage exists today to bet on the CloudFormation for AWS.
+
+### Direct Module Support
+
+Today, using a community module as a Radius recipe requires a wrapper that adds a `context` input and a structured `result` output conforming to Radius conventions. This wrapper adds friction, creates maintenance burden to republish to another IaC source and needs constant updates to stay in sync with upstream changes.
+
+Direct Module Support eliminates the Recipe wrapper for community modules. Platform engineers point `location` at any standard Bicep or Terraform module. Radius handles input resolution (injecting context like resource name and other resource properties into the module's native parameters via `{{context.*}}` expressions) and output resolution (mapping the module's native outputs to resource properties), all externally.
+
+The Recipe Pack bundles recipe definitions for multiple resource types. It maps each type to a module location, handles parameter injection via `{{context.*}}` expressions, and maps module outputs back to resource properties.
+
+```bicep
+// RecipePack resource definition
+resource recipepack 'Radius.Core/recipePacks@2025-08-01-preview' = {
+  name: 'azure-production'
+  properties: {
+    recipes: {
+      'Radius.Data/postgreSqlDatabases': {
+        kind: 'bicep'
+        location: 'mcr.microsoft.com/bicep/avm/res/db-for-postgre-sql/flexible-server:0.15.3'
+        parameters: {
+          name: 'pg-{{context.resource.name}}'
+        }
+        outputs: {
+          host: 'fqdn'
+          database: 'name'
+        }
+      }
+    }
+  }
+}
+
+// Environment references the recipe pack and provides location
+resource env 'Radius.Core/environments@2025-08-01-preview' = {
+  name: 'my-env'
+  properties: {
+    recipePacks: [
+      recipepack.id
+    ]
+    providers: {
+      azure: {
+        scope: '/subscriptions/.../resourceGroups/my-rg'
+      }
+      kubernetes: {
+        namespace: 'my-app-ns'
+      }
+    }
+  }
+}
+```
+
+For the full technical specification, see [Direct Recipe Modules](https://github.com/radius-project/radius/pull/11876).
+
+### Recipe Drivers Coverage
+
+Radius supports Bicep and Terraform drivers today. Bicep is the Azure-native path through AVM, while Terraform provides the broadest AWS and multi-cloud coverage.
+
+Stack Overflow 2025 places Terraform at 17.8% adoption across all respondents, behind Docker (71.1%) and Kubernetes (28.5%) but well ahead of other infrastructure tools like Ansible (11.7%). The survey no longer breaks out Bicep, CloudFormation, or Pulumi individually, having merged IaC into a broader "Cloud development" category.
+
+CNCF 2025 confirms Kubernetes at 82% production adoption among container users, with Helm at 81-87% adoption among Kubernetes organizations. That Helm number is what makes it the highest-leverage next driver for Radius: it maps directly to how the Kubernetes ecosystem already packages and distributes software, and unlocks existing charts instead of requiring custom Bicep modules per backing service.
+
+## Risks and Dependencies
+
+- **Upstream module quality**: We reference community modules we don't own. If AVM or a Terraform module ships a bad release or drops an output, our recipes break. We pin versions and run weekly CI to catch this early.
+- **Helm driver**: Kubernetes recipes are stuck on custom Bicep until we build the Helm driver. That's a bottleneck for K8s coverage.
+- **AVM gaps**: Some Tier 1 types don't have AVM modules yet. For those, we fall back to Terraform or custom Bicep on Azure.
+- **Schema stability**: Agents depend on resource type schemas. If we break a schema after agents are already using it, generated `app.bicep` files will be wrong. We enforce backward compatibility from Beta onward. 
+
+## Action Plan
+
+| Priority | Work Item |
+|----------|-----------|
+| **P0** | Fix deployment engine bugs with AVM modules |
+| **P0** | Direct Module Support |
+| **P0** | Build Tier 1 Resource type definitions and Recipes |
+| **P1** | Test framework for Resource Types and Recipes |
+| **P1** | Revamp contrib repository documentation with contribution gates |
+| **P1** | Build Tier 2 Resource type definitions and Recipes |
+| **P1** | Helm recipe driver |
+| **P2** | Recipe packs (dev-local, azure-prod, aws-prod) |
+| **P2** | Shared resource types (connection metadata, no recipe) |
+| **P3** | CloudFormation driver for native AWS recipe support |
+| **P3** | Build Tier 3 Resource type definitions and Recipes |
+
+## Success Metrics
+
+The primary measure is whether the resource type catalog is broad enough for AI agents to generate useful `app.bicep`.
+
+### Resource type and Recipe correctness
+
+- **Dependency resolution rate**: 80%+ of detected app dependencies should map to a defined resource type, measured across common application patterns (web apps, microservices, data pipelines).  
+- **Schema accuracy**: Generated `app.bicep` files should require zero manual edits for connection properties (host, port, credentials) when deploying against any supported platform.
+- **Deployment success rate**: 100% success rate across all module references, measured by weekly automated test runs against pinned module versions on all three platforms (Azure, AWS, Kubernetes).
+- **Breakage detection**: Upstream module changes that break a recipe should be caught with CI runs before they impact deployments.
+
+### Ecosystem Growth
+
+The contribution model covers types only, no recipe code. Success means external contributors find the process accessible and the type catalog grows steadily with community involvement.
+
+- Growing number of external contributions over time (10 per month)
+- Repeat contributors returning to add or improve types (at least 2 per month)
+- Type catalog expanding across all three tiers
+
+## References
+
+1. [Stack Overflow Developer Survey 2025](https://survey.stackoverflow.co/2025/)
+2. [CNCF Annual Cloud Native Survey 2025](https://www.cncf.io/wp-content/uploads/2026/01/CNCF_Annual_Survey_Report_final.pdf)
+
+---
+
+## Appendix: Testing, Validation, and Contribution Model
+
+This will be covered in another doc with the technical details of the testing framework and contribution process, but here are the high-level principles.
+
+### Testing and Validation
+
+**Schema Validation** ensures the resource type contract remains stable over time:
+
+- Required property and type checks
+- Output contract validation
+- Backward compatibility checks across schema versions
+
+**Recipe Validation** catches upstream module breakage before it impacts deployments:
+
+- Weekly CI runs that deploy each referenced module and verify outputs map correctly
+- Version-pinned references with automated PRs when new module versions are available
+- Cross-platform validation (Azure, AWS, Kubernetes)
+
+### Contributions, Ownership and Lifecycle
+
+The `resource-types-contrib` repository contains resource type definitions and tested recipe module references. Contributions include:
+
+1. Resource Type schema
+2. Documentation and module references (which AVM/TF/Helm modules to use)
+3. Tests that validate the type deploys correctly with those modules
+
+### Maturity Stages
+
+Contributions enter at Alpha or Beta and graduate towards Stable. Stable resource-types are what is added to Radius install.
+
+> **Note:** Resource types that exist in Radius today (e.g., Applications.Core/*, Applications.Data/*) are automatically promoted to Stable once Radius maintainers validate their extensibility equivalents.
+
+**Alpha**
+
+- Schema passes validation (required properties, type checks, output contract)
+- At least one working recipe module reference on any single platform
+- README with usage examples
+- Manual testing evidence submitted with the PR
+
+**Beta**
+
+- Recipe module references for all three platforms (AWS, Azure, Kubernetes)
+- README with usage examples across all platforms
+- Recipe packs tested across all supported platforms
+- Backward-compatible schema changes only
+
+**Stable**
+
+- 100% functional test coverage across all supported modules
+- Full CI/CD integration with release pipeline
+- Proven documentation validated by external community members
+
+**Radius maintainers** are responsible for schema evolution, versioning, compatibility guarantees, deprecation, and promotion through maturity stages.