Skip to content

Commit 7267c43

Browse files
authored
feat(m365): Use Managed Identity for Getting Application Certificate in Container (#6)
* use managed identity to access app certificate directly from container * update Terraform providers; change storage urls, add docs for setting subscrption_id * remove cert rotation * update readme variables and util file * fix apply loop with containers by setting ip type to private * add instruction for setting environment in provider.tf * change keyvault name on serial number change
1 parent f63305b commit 7267c43

18 files changed

Lines changed: 297 additions & 202 deletions

File tree

m365/README.adoc

Lines changed: 15 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -46,28 +46,31 @@ It is expected that this code is deployed by a user with administrator privledge
4646
[#deploy]
4747
=== Deploying with Terraform
4848

49+
. Run `az login` if not already done to configure your credentials
4950
. Prepare a directory for your deployment
5051
.. Change directories to `m365/terraform/env`
5152
.. Create a copy of the `example` directory with a name of your choice (e.g., `<myenv>`). **The remaining steps should be completed in this new directory**
5253
. Update variables and configurations
5354
.. In your new directory, `<myenv>`, modify the `variables.tfvars` file to configure the deployment for your needs
5455
... Set `contact_emails` to administrators' emails and set `resource_group_name` to the resource group to create and deploy infrastructure in
5556
... Review the defaults used for optional variables in <<terraform-variables>>. Some of these may need to be modified depending on your environment
56-
.. (Optional, but recommended) Modify the `provider.tf` file to configure Terraform to store state in Azure. See external https://developer.hashicorp.com/terraform/language/backend/azurerm[documentation]
57+
.. In `provider.tf`, replace `<YOUR_SUBSCRIPTION_UUID>` with the Subscription ID you'll be deploying in. The subscription ID can be found in the console, or as the `id` field output by the command `az account show`
58+
.. If you are deploying in GCC High, update `provider.tf` to replace the two instances of `environment = "public"` with `environment = "usgovernment"`
59+
.. (Optional, but recommended) Modify the `provider.tf` file to configure Terraform to store state in Azure. See external https://developer.hashicorp.com/terraform/language/backend/azurerm[documentation].
5760
. Run terraform
58-
.. Run `az login` if not already done to configure your credentials
5961
.. In your `<myenv>` directory, Run `terraform init`. This only needs to be done once unless providers are updated
6062
.. In your `<myenv>` directory, Run `terraform apply -var-file=variables.tfvars`. Confirm changes meet your expectations then type "yes"
6163
. Onboard a tenant following the guidance in <<onboard>>
6264

6365
.Example of completing steps 1-3 in bash
6466
[source,shell]
6567
----
68+
$ az login
6669
$ cd m365/terraform/env
6770
m365/terraform/env$ cp -r example myenv
6871
m365/terraform/env$ cd myenv
6972
m365/terraform/env/myenv$ vim variables.tfvars
70-
m365/terraform/env/myenv$ az login
73+
m365/terraform/env/myenv$ vim provider.tf
7174
m365/terraform/env/myenv$ terraform init # only needed once
7275
m365/terraform/env/myenv$ terraform apply -var-file=variables.tfvars
7376
----
@@ -77,9 +80,8 @@ m365/terraform/env/myenv$ terraform apply -var-file=variables.tfvars
7780
This section provides the description for all terraform variables sorted by their likelihood of being changed.
7881
For a typical deployment, set `contact_emails` and `resource_group_name` then review the defaults for the optional variables and override in the `tfvars` file as needed.
7982

80-
8183
Required::
82-
`contact_emails` (string) ::: Emails to notify for alerts and before certificate expiry
84+
`contact_emails` (list(string)) ::: Emails to notify for alerts and before certificate expiry
8385
`resource_group_name` (string) ::: Resource group to create and build resources in
8486
Optional::
8587
`location` (string) [default=East US]::: Region to build resources in
@@ -92,14 +94,14 @@ Optional::
9294
`serial_number` (string) [default=01]::: Increment by 1 when re-provisioning with the same resource group name
9395
`image_path` (string) [default=./cisa_logo.png]::: Path to image used for app logo. Displayed in Azure console on installed tenants
9496
Advanced::
95-
`certificate_rotation_period_days` (number) [default=30]::: How many days between when the certificate key should be rotated. Note: rotation requires running terraform
9697
`create_app` (bool) [default=True]::: If true, the app will be created. If false, the app will be imported
9798
`prefix_override` (string) [default=None]::: Prefix for resource names. If null, one will be generated from app_name
98-
`input_storage_container_id` (string) [default=None]::: If not null, input container to read configs from (must give permissions to service account). Otherwise by default will create storage container.
99-
`output_storage_container_id` (string) [default=None]::: If not null, output container to put results in (must give permissions to service account). Otherwise by default will create storage container.
99+
`input_storage_container_url` (string) [default=None]::: If not null, input container to read configs from (must give permissions to service account). Otherwise by default will create storage container. Expect an https url pointing to a container
100+
`output_storage_container_url` (string) [default=None]::: If not null, output container to put results in (must give permissions to service account). Otherwise by default will create storage container. Expect an https url pointing to a container
100101
`tenants_dir_path` (string) [default=./tenants]::: Relative path to directory containing tenant configuration files in yaml
101102
`container_registry` (object) [default={'server': 'ghcr.io'}]::: Credentials for logging into registry with container image
102103
`container_image` (string) [default=ghcr.io/cisagov/scubaconnect-m365:latest]::: Docker image to use for running ScubaGear.
104+
`container_memory_gb` (number) [default=3]::: Amount of memory to allocate for ScubaGear container. Due to memory leaks in some dependencies, this may need to be increased if running on many tenants
103105

104106
[#onboard]
105107
=== Onboarding a Tenant
@@ -114,7 +116,7 @@ It will perform the following steps (note that the required permissions/roles co
114116
. Register the ScubaConnect application as a PowerApps Admin
115117

116118

117-
Once completed, upload a ScubaGear configuration file to the `input_storage_container_id` named `<tenant_fqdn>.yaml` (e.g., `myorg.onmicrosoft.com.yaml`).
119+
Once completed, upload a ScubaGear configuration file to the `input_storage_container_url` named `<tenant_fqdn>.yaml` (e.g., `myorg.onmicrosoft.com.yaml`).
118120
You may upload the file directly to Azure, or place it in `env/<your_env>/tenants/` and run `terraform apply`.
119121
Refer to the https://github.com/cisagov/ScubaGear/blob/main/docs/configuration/configuration.md#scuba-compliance-use[ScubaGear Configuration File documentation] for details on creating the configuration file.
120122

@@ -133,13 +135,10 @@ Looks into the "Last Run Output" logs to determine the cause.
133135

134136
=== Maintenance
135137

136-
GearConnect's architecture (limited by Managed Identity support in Windows containers) requires exporting the app's certificate as a secret variable in the container.
137-
To mitigate this, the certificate is short-lived.
138-
Terraform is set up to automatically generate a new certificate every `certificate_rotation_period_days` (defaults to 30).
139-
To utilize this mechanism, you must run `terraform apply` on a regular basis.
140-
This can be done through scheduled CI/CD or manually (an email will be sent one week prior to expiration).
141-
This will ensure the certificate is always valid.
142-
138+
GearConnect utilizes a certificate to authenticate to tenants.
139+
This certificate is configured to expire after 1 year.
140+
60 days prior to expiration, an email will be sent.
141+
Running `terraform apply` will update the certificate for you.
143142

144143
The container will be regularly rebuilt and updated overtime to support new versions of ScubaGear.
145144
No action is required for container updates as Azure Container Instances will grab the latest image by default.

m365/image/run_container.ps1

Lines changed: 39 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,47 @@
22
$Host.UI.RawUI.BufferSize = New-Object Management.Automation.Host.Size (500, 25)
33
$ErrorActionPreference = "Stop"
44

5+
if ($Env:DEBUG_LOG -eq "true") {
6+
Get-ChildItem env:
7+
}
8+
9+
Write-Output "Getting certificate from keyvault"
10+
# Retrieve an Access Token
11+
if (($Env:IS_VNET -eq "true") -and $Env:IDENTITY_ENDPOINT -like "http://10.92.0.*:2377/metadata/identity/oauth2/token?api-version=1.0") {
12+
$identityEndpoint = "http://169.254.128.1:2377/metadata/identity/oauth2/token?api-version=1.0"
13+
} else {
14+
$identityEndpoint = $Env:IDENTITY_ENDPOINT
15+
}
16+
17+
if ($Env:IS_GOV -eq "true") {
18+
$VaultURL = "https://$($Env:VAULT_NAME).vault.usgovcloudapi.net"
19+
$RawVaultURL = "https%3A%2F%2F" + "vault.usgovcloudapi.net"
20+
}
21+
else {
22+
$VaultURL = "https://$($Env:VAULT_NAME).vault.azure.net"
23+
$RawVaultURL = "https%3A%2F%2F" + "vault.azure.net"
24+
}
25+
26+
$uri = $identityEndpoint + '&resource=' + $RawVaultURL + '&principalId=' + $Env:MI_PRINCIPAL_ID
27+
$headers = @{
28+
secret = $Env:IDENTITY_HEADER
29+
"Content-Type" = "application/x-www-form-urlencoded"
30+
}
31+
32+
$response = Invoke-RestMethod -Uri $uri -Headers $headers -Method Get
33+
34+
# Access values from Key Vault with token
35+
$accessToken = $Response.access_token
36+
$headers2 = @{
37+
Authorization = "Bearer $accessToken"
38+
}
39+
40+
$PrivKey = (Invoke-RestMethod -Uri "$($VaultURL)/Secrets/$($Env:CERT_NAME)/?api-version=7.4" -Headers $headers2).Value
41+
$PFX_BYTES = [Convert]::FromBase64String($PrivKey)
542
Write-Output "Installing cert"
643
# Install certificate by decoding env variable
744
$PFX_FILE = '.\certificate.pfx'
8-
$BYTES = [Convert]::FromBase64String($Env:PFX_B64)
9-
[IO.File]::WriteAllBytes($PFX_FILE, $BYTES)
45+
[IO.File]::WriteAllBytes($PFX_FILE, $PFX_BYTES)
1046
$CertificateThumbPrint = (Import-PfxCertificate -FilePath $PFX_FILE -CertStoreLocation cert:\CurrentUser\My).Thumbprint
1147
Write-Output " CERT: $CertificateThumbPrint"
1248

@@ -16,6 +52,7 @@ $Env:AZCOPY_SPA_APPLICATION_ID= $Env:APP_ID
1652
$Env:AZCOPY_TENANT_ID=$Env:TENANT_ID
1753
$Env:AZCOPY_AUTO_LOGIN_TYPE="SPN"
1854
$Env:AZCOPY_SPA_CERT_PATH=$PFX_FILE
55+
$Env:AZCOPY_ACTIVE_DIRECTORY_ENDPOINT = if ($Env:IS_GOV -eq "true") {"https://login.microsoftonline.us"} else {"https://login.microsoftonline.com"}
1956

2057
# Print scuba version to console for debugging
2158
Invoke-SCuBA -Version

m365/terraform/env/example/main.tf

Lines changed: 16 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,18 @@
11
module "scuba_connect" {
2-
source = "../.."
3-
app_name = var.app_name
4-
app_multi_tenant = var.app_multi_tenant
5-
image_path = var.image_path
6-
contact_emails = var.contact_emails
7-
resource_group_name = var.resource_group_name
8-
serial_number = var.serial_number
9-
location = var.location
10-
schedule_interval = var.schedule_interval
11-
tenants_dir_path = var.tenants_dir_path
12-
vnet = var.vnet
13-
container_image = var.container_image
14-
container_registry = var.container_registry
15-
input_storage_container_id = var.input_storage_container_id
16-
output_storage_container_id = var.output_storage_container_id
17-
certificate_rotation_period_days = var.certificate_rotation_period_days
18-
tags = var.tags
2+
source = "../.."
3+
app_name = var.app_name
4+
app_multi_tenant = var.app_multi_tenant
5+
image_path = var.image_path
6+
contact_emails = var.contact_emails
7+
resource_group_name = var.resource_group_name
8+
serial_number = var.serial_number
9+
location = var.location
10+
schedule_interval = var.schedule_interval
11+
tenants_dir_path = var.tenants_dir_path
12+
vnet = var.vnet
13+
container_image = var.container_image
14+
container_registry = var.container_registry
15+
input_storage_container_url = var.input_storage_container_url
16+
output_storage_container_url = var.output_storage_container_url
17+
tags = var.tags
1918
}

m365/terraform/env/example/outputs.tf

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,14 @@ output "app_id" {
33
value = module.scuba_connect.app_id
44
}
55

6-
output "output_storage_container_id" {
7-
description = "ID of the output storage account results are written to"
8-
value = module.scuba_connect.output_storage_container_id
6+
output "output_storage_container_url" {
7+
description = "URL of the output storage account results are written to"
8+
value = module.scuba_connect.output_storage_container_url
99
}
1010

11-
output "input_storage_container_id" {
12-
description = "ID of the input storage account configs are read from"
13-
value = module.scuba_connect.output_storage_container_id
11+
output "input_storage_container_url" {
12+
description = "URL of the input storage account configs are read from"
13+
value = module.scuba_connect.output_storage_container_url
1414
}
1515

1616
output "sp_object_id" {

m365/terraform/env/example/provider.tf

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,11 @@ terraform {
33
required_providers {
44
azurerm = {
55
source = "hashicorp/azurerm"
6-
version = "~> 3.98.0"
6+
version = "~> 4.22.0"
77
}
88
azuread = {
99
source = "hashicorp/azuread"
10-
version = "2.47.0"
10+
version = "~> 3.1.0"
1111
}
1212
}
1313

@@ -16,8 +16,10 @@ terraform {
1616

1717
provider "azurerm" {
1818
features {}
19+
subscription_id = "<YOUR_SUBSCRIPTION_UUID>"
20+
environment = "public"
1921
}
2022

2123
provider "azuread" {
22-
24+
environment = "public"
2325
}

m365/terraform/env/example/variables.tf

Lines changed: 4 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -81,16 +81,6 @@ variable "image_path" {
8181

8282
### ADVANCED ###
8383

84-
variable "certificate_rotation_period_days" {
85-
type = number
86-
description = "How many days between when the certificate key should be rotated. Note: rotation requires running terraform"
87-
default = 30
88-
validation {
89-
condition = var.certificate_rotation_period_days <= 60 && var.certificate_rotation_period_days >= 3
90-
error_message = "Rotation period must be between 3 and 60 days"
91-
}
92-
}
93-
9484
variable "create_app" {
9585
default = true
9686
type = bool
@@ -103,16 +93,16 @@ variable "prefix_override" {
10393
description = "Prefix for resource names. If null, one will be generated from app_name"
10494
}
10595

106-
variable "input_storage_container_id" {
96+
variable "input_storage_container_url" {
10797
default = null
10898
type = string
109-
description = "If not null, input container to read configs from (must give permissions to service account). Otherwise by default will create storage container."
99+
description = "If not null, input container to read configs from (must give permissions to service account). Otherwise by default will create storage container. Expect an https url pointing to a container"
110100
}
111101

112-
variable "output_storage_container_id" {
102+
variable "output_storage_container_url" {
113103
default = null
114104
type = string
115-
description = "If not null, output container to put results in (must give permissions to service account). Otherwise by default will create storage container."
105+
description = "If not null, output container to put results in (must give permissions to service account). Otherwise by default will create storage container. Expect an https url pointing to a container"
116106
}
117107

118108
variable "tenants_dir_path" {

m365/terraform/main.tf

Lines changed: 33 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -25,23 +25,7 @@ resource "azurerm_log_analytics_workspace" "monitor_law" {
2525
lifecycle {
2626
ignore_changes = [tags]
2727
}
28-
depends_on = [ azurerm_resource_group_policy_assignment.tagging_assignments ]
29-
}
30-
31-
# Creates the app registration, or reads an existing one, which is used by the ScubaGear container
32-
module "app" {
33-
source = "./modules/app"
34-
resource_group_name = azurerm_resource_group.rg.name
35-
location = var.location
36-
resource_prefix = local.name
37-
app_name = var.app_name
38-
image_path = var.image_path
39-
create_app = var.create_app
40-
contact_emails = var.contact_emails
41-
allowed_access_ips = try(var.vnet.allowed_access_ip_list, null)
42-
certificate_rotation_period_days = var.certificate_rotation_period_days
43-
app_multi_tenant = var.app_multi_tenant
44-
depends_on = [azurerm_resource_group_policy_assignment.tagging_assignments]
28+
depends_on = [azurerm_resource_group_policy_assignment.tagging_assignments]
4529
}
4630

4731
module "networking" {
@@ -55,23 +39,38 @@ module "networking" {
5539
depends_on = [azurerm_resource_group_policy_assignment.tagging_assignments]
5640
}
5741

42+
# Creates the app registration, or reads an existing one, which is used by the ScubaGear container
43+
module "app" {
44+
source = "./modules/app"
45+
resource_group_name = azurerm_resource_group.rg.name
46+
location = var.location
47+
kv_prefix = "${local.name}-${var.serial_number}"
48+
app_name = var.app_name
49+
image_path = var.image_path
50+
create_app = var.create_app
51+
contact_emails = var.contact_emails
52+
allowed_access_ips = try(var.vnet.allowed_access_ip_list, null)
53+
aci_subnet_id = try(module.networking[0].aci_subnet_id, null)
54+
app_multi_tenant = var.app_multi_tenant
55+
depends_on = [azurerm_resource_group_policy_assignment.tagging_assignments]
56+
}
5857

5958
module "container" {
60-
source = "./modules/container"
61-
resource_prefix = local.name
62-
resource_group = azurerm_resource_group.rg
63-
container_registry = var.container_registry
64-
container_image = var.container_image
65-
application_client_id = module.app.client_id
66-
application_object_id = module.app.sp_object_id
67-
application_pfx_b64 = module.app.certificate_pfx_b64
68-
allowed_access_ips = try(var.vnet.allowed_access_ip_list, null)
69-
subnet_ids = var.vnet == null ? null : [module.networking[0].aci_subnet_id]
70-
schedule_interval = var.schedule_interval
71-
output_storage_container_id = var.output_storage_container_id
72-
input_storage_container_id = var.input_storage_container_id
73-
contact_emails = var.contact_emails
74-
log_analytics_workspace = azurerm_log_analytics_workspace.monitor_law
75-
container_memory_gb = var.container_memory_gb
76-
depends_on = [azurerm_resource_group_policy_assignment.tagging_assignments]
59+
source = "./modules/container"
60+
resource_prefix = local.name
61+
resource_group = azurerm_resource_group.rg
62+
container_registry = var.container_registry
63+
container_image = var.container_image
64+
application_client_id = module.app.client_id
65+
application_object_id = module.app.sp_object_id
66+
allowed_access_ips = try(var.vnet.allowed_access_ip_list, null)
67+
subnet_ids = var.vnet == null ? null : [module.networking[0].aci_subnet_id]
68+
schedule_interval = var.schedule_interval
69+
output_storage_container_url = var.output_storage_container_url
70+
input_storage_container_url = var.input_storage_container_url
71+
contact_emails = var.contact_emails
72+
log_analytics_workspace = azurerm_log_analytics_workspace.monitor_law
73+
container_memory_gb = var.container_memory_gb
74+
cert_info = module.app.cert_info
75+
depends_on = [azurerm_resource_group_policy_assignment.tagging_assignments]
7776
}

0 commit comments

Comments
 (0)