Support azure-devops-oidc for serviceConnection name instead of pipeline name

[abij]

We encountered an issue that the Federation Policy for AzureDevOps must be based on the pipeline name.

Current working situation

I think we should get a medal for getting the azure-devops-oidc actually working.

When you read this, you are also wondering how to get the org-id . You cannot find this anyware in the Azure DevOps interface, but simple solution is to run a build-pipeline with System Diagnostics enabled and look for the property: SYSTEM_COLLECTIONID, yes thats the one you need! Using this ID you can created the Federation Policy for Databricks Account:

Content of my oidc_policy.json:

{
  "oidc_policy": {
    "issuer": "https://vstoken.dev.azure.com/<org GUID>",
    "audiences": ["api://AzureADTokenExchange"],
    "subject": "p://<Org name>/<Project>/<Pipeline name>"
  }
}

And executing using:
databricks account service-principal-federation-policy create <databricks-id-of-spn> --json @oidc_policy.json

Requested way of working

We would like to have a policy based on the name of the ServiceConnection sc, not on the name of the pipeline p. This way we can add a federation policy looking like this:

Content of my oidc_policy.json:

{
  "oidc_policy": {
    "issuer": "https://vstoken.dev.azure.com/<org GUID>",
    "audiences": ["api://AzureADTokenExchange"],
    "subject": "sc://<Org name>/<Project>/<ServiceConnection name>"
  }
}

I think it's a matter of adding the optional property serviceConnectionId as referenced here: https://learn.microsoft.com/en-us/rest/api/azure/devops/distributedtask/oidctoken/create?view=azure-devops-rest-7.1.

The code which should be updated:

databricks-sdk-go/config/experimental/auth/oidc/azuredevops.go

Line 83 in fd4721f

requestUrl := fmt.Sprintf("%s/%s/_apis/distributedtask/hubs/%s/plans/%s/jobs/%s/oidctoken?api-version=7.2-preview.1",

[neilmca-mks]

+1 for this

Major challenge in setting this up - the docs casually mention "just enter the Azure DevOps OrgID GUID" without telling you how to find it - that's your first day challenge sorted.

Got it working - then finding out that you can only create a maximum of 20 Federation Policies per Databricks Service Principal and then suddenly this all becomes even more unworkable. https://learn.microsoft.com/en-us/azure/databricks/dev-tools/auth/oauth-federation-policy#configure-a-service-principal-federation-policy

This would mean only 20 Azure DevOps pipelines could use that Databricks Service Principal in this regard.

It may be the recommended approach to follow, but in classic tech terms it doesn't actually fit how customers will actually have to struggle to use it. As it stands currently the solution here is not fit for purpose and requires a massive rethink

[neilmca-mks]

@abij I gave up on this and went a different route for what I needed to do - I'm using Azure Databricks so I can use Azure CLI to authenticate as an Azure DevOps Service Connection (set up as a type of Azure Resource Manager and using Workload Identity Federation) to have the ability to run Databricks CLI commands.

This is about as simple as I can make it given the other implementations severe limitations right now. To be honest it's using Workload Identity Federation now anyway, and it fits my current purpose - running Databricks CLI commands in an Azure DevOps pipeline. With this I don't have to create another Databricks Service Principal, or create a corresponding Federation Policy for it.

I'll paste my AZDO Yaml here in-case it is of any use to you or anyone else - Cheers

# Azure DevOps Pipeline YAML to authenticate to a Databricks Workspace - Neil McAlister - https://www.linkedin.com/in/neilmcalister/
# Feb 2026
#
# This pipeline will not perform any destructive operation inside the Databricks Workspace itself - merely just output some JSON on the current authenticated user
# It will use Azure CLI to authenticate and use that to allow running of the Databricks CLI commands thereafter
#
# The Azure DevOps Service Connection you create here can either be the Azure Resource Manager > App Registration (automatic) and Workload identity federation, or a SECRET method - it does not matter
# Providing the Azure CLI authenticates successfully it doesn't matter which option you choose
#
# Ensure you add this Azure DevOps Service Connection's Service Principal (AppReg) to the Databricks Workspace via SETTINGS > Identity and access > Service Principals - it will show as type EXTERNAL

trigger: none

pool:
  vmImage: 'ubuntu-latest'

variables:
  azureSubscription: '????????' # Your Azure DevOps Service Connection NAME
  DATABRICKS_HOST: 'https://????????.azuredatabricks.net/' # Your Databricks Workspace URL

steps:
  - script: |
      curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sh
    displayName: 'Install Databricks CLI on Microsoft Azure VM image ubuntu-latest'
  - task: AzureCLI@2
    displayName: 'Databricks Commands'
    inputs:
      azureSubscription: $(azureSubscription)   
      useGlobalConfig: true
      scriptType: bash
      scriptLocation: inlineScript
      inlineScript: |
        export DATABRICKS_HOST=$(DATABRICKS_HOST)
        databricks current-user me