Get Azure AD tokens for service principals
Important
This section describes how to manually get Azure AD tokens for service principals.
Databricks does not recommend that you create Azure AD tokens for Azure AD service principals manually. This is because each Azure AD token is short-lived, typically expiring within one hour. After this time, you must manually generate a replacement Azure AD token. Instead, use one of the participating tools or SDKs that implement the Databricks client unified authentication standard. These tools and SDKs automatically generate and replace expired Azure AD tokens for you, leveraging the following Databricks authentication types:
If you do not have a service principal, you can provision one by following one of these sets of instructions:
This article describes how a service principal defined in Azure Active Directory (Azure AD) can also act as a principal on which authentication and authorization policies can be enforced in Azure Databricks. Service principals in an Azure Databricks workspace can have different fine-grained access control than regular users (user principals).
A service principal acts as a client role and uses the OAuth 2.0 client credentials flow to authorize access to Azure Databricks resources.
You can manage service principals by using the Databricks SCIM API or by using the following procedure from the Azure portal.
You can also use the Microsoft Authentication Library (MSAL) to programmatically get an Azure AD access token for a user instead of a service principal. See Get Azure AD tokens for users by using MSAL.
Provision a service principal in Azure portal
Sign in to the Azure portal.
Note
The portal to use is different depending on whether your Azure AD application runs in the Azure public cloud or in a national or sovereign cloud. For more information, see National clouds.
If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to provision the service principal.
Search for and select Azure Active Directory.
Click + Add and select App registration.
For Name, enter a name for the application.
In the Supported account types section, select Accounts in this organizational directory only (Single tenant).
Click Register.
Within Manage, click Certificates & secrets.
On the Client secrets tab, click New client secret.

In the Add a client secret pane, for Description, enter a description for the client secret.
For Expires, select an expiry time period for the client secret, and then click Add.
Copy and store the client secret’s Value in a secure place, as this client secret is the password for your application.
On the application page’s Overview page, in the Essentials section, copy the following values:
- Application (client) ID
- Directory (tenant) ID

Provision a service principal with the Azure CLI
See Create an Azure service principal with the Azure CLI.
Get an Azure AD access token with the Microsoft identity platform REST API
Important
This section describes how to manually get an Azure AD token for a service principal by using the Microsoft identity platform REST API.
Databricks does not recommend that you create Azure AD tokens for Azure AD service principals manually. This is because each Azure AD token is short-lived, typically expiring within one hour. After this time, you must manually generate a replacement Azure AD token. Instead, use one of the participating tools or SDKs that implement the Databricks client unified authentication standard. These tools and SDKs automatically generate and replace expired Azure AD tokens for you, leveraging the following Databricks authentication types:
To access the Databricks REST API with the service principal, you get and then use an Azure AD access token for the service principal. For more information, see First case: Access token request with a shared secret.
Tip
You can also use the Azure CLI to get the Azure AD access token. See Get an Azure AD access token with the Azure CLI.
Gather the following information:
Parameter Description Tenant ID The Directory (tenant) ID for the application registered in Azure AD. Client ID The Application (client) ID for the application registered in Azure AD. Client secret The Value of the client secret for the application registered in Azure AD. Use the preceding information along with curl to get the Azure AD access token.
curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \ https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/token \ -d 'client_id=<client-id>' \ -d 'grant_type=client_credentials' \ -d 'scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default' \ -d 'client_secret=<client-secret>'Replace:
<tenant-id>with the registered application’s tenant ID.<client-id>with the registered application’s client ID.<client-secret>with the registered application’s client secret value.
Do not change the value of the
scopeparameter. It represents the programmatic ID for Azure Databricks (2ff814a6-3304-4ab8-85cb-cd0e6f879c1d) along with the default scope (/.default, URL-encoded as%2f.default).For example:
curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \ https://login.microsoftonline.com/a1bc2d34-5e67-8f89-01ab-c2345d6c78de/oauth2/v2.0/token \ -d 'client_id=12a34b56-789c-0d12-e3fa-b456789c0123' \ -d 'grant_type=client_credentials' \ -d 'scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default' \ -d 'client_secret=abc1D~Ef...2ghIJKlM3'The Azure AD access token is in the
access_tokenvalue within the output of the call.
Get an Azure AD access token with the Azure CLI
Important
This section describes how to manually get an Azure AD token for a service principal by using the Azure CLI.
Databricks does not recommend that you create Azure AD tokens for Azure AD service principals manually. This is because each Azure AD token is short-lived, typically expiring within one hour. After this time, you must manually generate a replacement Azure AD token. Instead, use one of the participating tools or SDKs that implement the Databricks client unified authentication standard. These tools and SDKs automatically generate and replace expired Azure AD tokens for you, leveraging the following Databricks authentication types:
To access the Databricks REST API with the service principal, you get and then use an Azure AD access token for the service principal.
Gather the following information:
Parameter Description Tenant ID The Directory (tenant) ID for the application registered in Azure AD. Client ID The Application (client) ID for the application registered in Azure AD. Client secret The Value of the client secret for the application registered in Azure AD. Sign in to Azure by using the Azure CLI to run the az login command. Use the
--service-principaloption along with specifying the values for the parameters of Tenant ID (Directory (tenant) ID), Client ID (Application (client) ID), and Client secret (Value) for the application registered in Azure AD. Use the--outputoption to display the command’s output in a different format (in this case, a table instead of JSON) for better readability.az login \ --service-principal \ --tenant <Tenant-ID> --username <Client-ID> \ --password <Client-secret> \ --output tableConfirm that you are signed in to the correct subscription for which you want to create the Azure AD access token for the signed-in Azure AD service principal. To do this, in the output of the
az logincommand, look for the table row whereIsDefaultis set toTrue. The subscription’sNameindicates the current subscription.If you are not sure what the correct subscription should be, see Get subscription and tenant IDs in the Azure portal. Or, try running the
az databricks workspace listcommand later in this step.If you need to switch to a different subscription, run the az account set command, using the
--nameor--subscriptionoption to specify the correct subscription name or ID.az account set --name "<subscription-name>" # Or ... az account set --subscription <subscription-id>You can get the subscription ID for an Azure Databricks workspace by clicking Azure Portal in the workspace navigation bar and looking for
Resource ID: /subscriptions/00000000-0000-0000-0000-000000000000, where00000000-0000-0000-0000-000000000000is the subscription ID.If you do not have access to the Azure Portal, you can try getting the subscription ID for an Azure Databricks workspace by running the az databricks workspace list command, using the
--queryand--outputoptions to narrow down the results. Replaceadb-0000000000000000.0.azuredatabricks.netwith the name of your workspace instance, not including thehttps://. In this example, the00000000-0000-0000-0000-000000000000after/subscriptions/in the output is the subscription ID.az databricks workspace list --query "[?workspaceUrl==\`adb-0000000000000000.0.azuredatabricks.net\`].{id:id}" --output tsv # /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-rg/providers/Microsoft.Databricks/workspaces/my-wsIf the following message displays, you are signed in to the wrong tenant:
The subscription of '<subscription-id>' doesn't exist in cloud 'AzureCloud'.To sign in to the correct tenant, you must run theaz logincommand again, using the--tenantoption to specify the correct tenant ID.You can the tenant ID for an Azure Databricks workspace by running the command
curl -v <per-workspace-URL>/aad/authand looking in the output< location: https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000, where00000000-0000-0000-0000-000000000000is the tenant ID. See also Get subscription and tenant IDs in the Azure portal.az login --tenant <tenant-id> --output tableGenerate the Azure AD access token for the signed-in Azure AD service principal by running the az account get-access-token command. Use the
--resourceoption to specify the unique resource ID for the Azure Databricks service, which is2ff814a6-3304-4ab8-85cb-cd0e6f879c1d. You can display just the Azure AD token’s value in the output of the command by using the--queryand--outputoptions.az account get-access-token \ --resource 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d \ --query "accessToken" \ --output tsv
Use the service principal’s Azure AD access token to access the Databricks REST API
Important
This section describes how to use curl and a service principal’s Azure AD access token to access the Databricks REST API.
Instead of curl, you might want to use one of the participating tools or SDKs that implement the Databricks client unified authentication standard instead. These tools and SDKs enable you to use popular tools such as Terraform and Visual Studio Code, and popular programming languages such as Python and Java. These tools and SDKs also automatically generate and replace expired Azure AD tokens for you, leveraging the following Databricks authentication types:
A service principal that is a Databricks user can authenticate to the Databricks REST API with Azure Active Directory tokens.
A service principal can also add itself as a workspace admin to a workspace if it has the Contributor or Owner role on the target workspace resource in Azure. If the service principal is a Contributor or Owner on the target workspace and you want to add it to a workspace using its Azure Active Directory token, go to Workspace-level API access for service principals that are not Azure Databricks users.
Otherwise, continue to API access for service principals that are Azure Databricks users and admins.
API access for service principals that are Azure Databricks users and admins
To complete this procedure, you must first add the service principal to the Azure Databricks account or workspace. You can add your service principal directly to your account, without granting it workspace access, by using the SCIM (Account) API.
You can add your service principal to a workspace by using the Service Principals API endpoint. This will also add the service principal to your Azure Databricks account. For example:
curl -X POST \
-H 'Authorization: Bearer <access-token>' \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/preview/scim/v2/ServicePrincipals \
-H 'Content-type: application/scim+json' \
-d @create-service-principal.json
create-service-principal.json:
{
"displayName": "My Service Principal",
"applicationId": "12a34b56-789c-0d12-e3fa-b456789c0123",
"entitlements": [
{
"value": "allow-cluster-create"
}
],
"schemas": [
"urn:ietf:params:scim:schemas:core:2.0:ServicePrincipal"
],
"active": true
}
Skip ahead to Workspace-level API access for service principals that are not Azure Databricks users if either of the following are true:
- The Azure Databricks REST API that you want to call requires workspace admin access and the service principal is a member of the workspace, but does not currently have admin access to the workspace.
- The service principal has not already been added to the target Azure Databricks workspace.
Gather the following information.
Parameter Description Azure AD access token The Azure AD access token returned from the request in Get an Azure AD access token with the Microsoft identity platform REST API or Get an Azure AD access token with the Azure CLI. Use the Azure AD access token along with
curlto call the Databricks REST API. For example:curl -X GET \ -H 'Authorization: Bearer <access-token>' \ https://<databricks-instance>/api/2.0/clusters/listReplace:
<access-token>with the Azure AD access token.<databricks-instance>with the per-workspace URL of your Azure Databricks deployment.GETand/api/2.0/clusters/listwith the appropriate HTTP operation and endpoint for the target Databricks REST API.
For example:
curl -X GET \ -H 'Authorization: Bearer <access-token>' \ https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/clusters/list
Workspace-level API access for service principals that are not Azure Databricks users
Follow this procedure if any of the following are true:
- The Azure Databricks REST API that you want to call requires workspace admin access and the service principal is a member of the workspace, but does not currently have admin access to the workspace.
- The service principal has not already been added to the target Azure Databricks workspace.
Requirements:
- The service principal requires the Contributor or Owner role on the target workspace resource in Azure.
Gather the following information:
Parameter Description Tenant ID The Directory (tenant) ID for the application registered in Azure AD in Provision a service principal in Azure portal. Client ID The Application (client) ID for the application registered in Azure AD. Client secret The Value of the client secret for the application registered in Azure AD, which you created in Provision a service principal in Azure portal. Azure AD access token The Azure AD access token returned from the request in Get an Azure AD access token with the Microsoft identity platform REST API or Get an Azure AD access token with the Azure CLI. Subscription ID The ID (not the name) of the Azure subscription that is associated with the target Azure Databricks workspace. To get to this and the following information, see Open resources. To open the target resource, you can search on the Azure Databricks service type and any other information in Azure that you know about the target Azure Databricks workspace. Resource group name The name of the Azure resource group that is associated with the target Azure Databricks workspace. Workspace name The name in Azure of the target Azure Databricks workspace. Use some of the preceding information along with
curlto get an Azure AD management endpoint access token.curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \ https://login.microsoftonline.com/<tenant-id>/oauth2/token \ -d 'client_id=<client-id>' \ -d 'grant_type=client_credentials' \ -d 'resource=https%3A%2F%2Fmanagement.core.windows.net%2F' \ -d 'client_secret=<client-secret>'Replace:
<tenant-id>with the registered application’s tenant ID.<client-id>with the registered application’s client ID.<client-secret>with the registered application’s client secret value.
Do not change the value of the
resourceparameter. It represents the Azure AD management endpoint (https://management.core.windows.net/, URL-encoded ashttps%3A%2F%2Fmanagement.core.windows.net%2F).For example:
curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \ https://login.microsoftonline.com/a1bc2d34-5e67-8f89-01ab-c2345d6c78de/oauth2/token \ -d 'client_id=12a34b56-789c-0d12-e3fa-b456789c0123' \ -d 'grant_type=client_credentials' \ -d 'resource=https%3A%2F%2Fmanagement.core.windows.net%2F' \ -d 'client_secret=abc1D~Ef...2ghIJKlM3'The Azure AD management endpoint access token is in the
access_tokenvalue within the output of the call.Use the Azure AD management endpoint access token along with the rest of the preceding information and
curlto call the Databricks REST API, for example:curl -X GET \ -H 'Authorization: Bearer <access-token>' \ -H 'X-Databricks-Azure-SP-Management-Token: <management-access-token>' \ -H 'X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Databricks/workspaces/<workspace-name>' \ https://<databricks-instance>/api/2.0/clusters/listReplace:
<access-token>with the Azure AD access token.<management-access-token>with the Azure AD management endpoint access token.<subscription-id>with the ID of the subscription that is associated with the target Azure Databricks workspace.<resource-group-name>with the name of the resource group that is associated with the target Azure Databricks workspace.<workspace-name>with the name of the target Azure Databricks workspace.<databricks-instance>with the per-workspace URL of your Azure Databricks deployment.GETand/api/2.0/clusters/listwith the appropriate HTTP operation and endpoint for the target Databricks REST API.
For example:
curl -X GET \ -H 'Authorization:Bearer <access-token>' \ -H 'X-Databricks-Azure-SP-Management-Token: abC1dE...ghIj23kl' \ -H 'X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/12a345...bcd6789e/resourceGroups/my-resource-group/providers/Microsoft.Databricks/workspaces/my-workspace' \ https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/clusters/list
Upon authenticating to the workspace, the service principal becomes an Azure Databricks workspace admin and no longer needs the Contributor or Owner role to access the workspace.
Feedback
Submit and view feedback for