Manage service principals
This article explains how to create and manage service principals for your Azure Databricks account and workspaces.
For an overview of the Azure Databricks identity model, see Azure Databricks identities and roles.
What is a service principal?
A service principal is an identity that you create in Azure Databricks for use with automated tools, jobs, and applications. Service principals give automated tools and scripts API-only access to Azure Databricks resources, providing greater security than using users or groups.
You can grant and restrict a service principal’s access to resources in the same way as you can an Azure Databricks user. For example, you can do the following:
- Give a service principal account admin and workspace admin roles.
- Give a service principal access to data, either at the account level using Unity Catalog, or at the workspace level.
- Add a service principal to a group at both the account and workspace level, including the workspace
adminsgroup.
You can also grant Azure Databricks users, service principals, and groups permissions to use a service principal. This allows users to run jobs as the service principal, instead of as their identity. This prevents jobs from failing if a user leaves your organization or a group is modified.
Unlike an Azure Databricks user, a service principal is an API-only identity; it cannot be used to access the Azure Databricks UI.
Azure Databricks recommends that you enable your workspaces for identity federation so that you can manage your service principals in the account. If your workspace isn’t enabled for identity federation, you can create and manage service principals using the workspace-level SCIM APIs.
Add a service principal to your Azure Databricks account
Account admins can add service principals to your Azure Databricks account using the account console or the SCIM (Account) API.
To use service principals on Azure Databricks, an admin user must create a new Azure Active Directory (Azure AD) application and then add it to the Azure Databricks workspace to use as a service principal. To create an Azure AD service principal, follow these instructions:
Sign in to the Azure portal.
Note
The portal to use is different depending on whether your Azure AD application runs in the Azure public cloud or in a national or sovereign cloud. For more information, see National clouds.
If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to provision the service principal.
Search for and select Azure Active Directory.
Click + Add and select App registration.
For Name, enter a name for the application.
In the Supported account types section, select Accounts in this organizational directory only (Single tenant).
Click Register.
Within Manage, click Certificates & secrets.
On the Client secrets tab, click New client secret.

In the Add a client secret pane, for Description, enter a description for the client secret.
For Expires, select an expiry time period for the client secret, and then click Add.
Copy and store the client secret’s Value in a secure place, as this client secret is the password for your application.
On the application page’s Overview page, in the Essentials section, copy the following values:
- Application (client) ID
- Directory (tenant) ID

Add service principals to your account using the account console
To add a service principal to the account using the account console:
- As an account admin, log in to the account console.
- In the sidebar, click User management.
- On the Service principals tab, click Add service principal.
- Enter a name for the service principal.
- Under UUID, paste the Application (client) ID for the service principal.
- Click Add.
Add service principals to your account using the SCIM (Account) API
Account admins can add and manage service principals in the Azure Databricks account using the SCIM API for Accounts.
Workspace admins can also create and manage service principals using this API, but they must invoke the API using a different endpoint URL:
- Account admins use
accounts.azuredatabricks.net/api/2.0/accounts/{account_id}/scim/v2/. - Workspace admins use
{workspace-domain}/api/2.0/account/scim/v2/.
For details, see the Account Groups API.
Assign account admin rights to a service principal
To assign account admin rights using the account console, do the following:
- As an account admin, log in to the account console.
- In the sidebar, click User management.
- On the Service principals tab, find and click the username.
- On the Roles tab, turn on Account admin.
You can also assign the account admin role using the Groups API.
To use service principals as account admins, you must generate Azure AD tokens for them. See Get Azure AD tokens for service principals.
Remove service principals from your Azure Databricks account
Account admins can delete service principals from an Azure Databricks account. Workspace admins cannot. When you delete a service principal from the account, that principal is also removed from their workspaces.
Important
When you remove a service principal from the account, that service principal is also removed from their workspaces, regardless of whether or not identity federation has been enabled. We recommend that you refrain from deleting account-level service principals unless you want them to lose access to all workspaces in the account. Be aware of the following consequences of deleting service principals:
- Applications or scripts that use the tokens generated by the service principal will no longer be able to access the Databricks API
- Jobs owned by the service principal will fail
- Clusters owned by the service principal will stop
- Queries or dashboards created by the service principal and shared using the Run as Owner credential will have to be assigned to a new owner to prevent sharing from failing
To remove a service principal using the account console, do the following:
- As an account admin, log in to the account console.
- In the sidebar, click User management.
- On the Service principals tab, find and click the username.
- On the Principal Information tab, click the
kebab menu in the upper-right corner and select Delete. - In the confirmation dialog box, click Confirm delete.
Add a service principal to a workspace
Account admins can add service principals to identity federated workspaces using the account console and the Workspace Assignment API.
Workspace admins can manage service principals in their workspaces using the workspace admin settings page, the Workspace Assignment API, and the workspace-level SCIM (ServicePrincipals) API.
Assign a service principal to a workspace using the account console
- As an account admin, log in to the account console.
- In the sidebar, click Workspaces.
- Click your workspace name.
- On the Permissions tab, click Add permissions.
- Search for and select the service principal, assign the permission level (workspace User or Admin), and click Save.
Add a service principal to a workspace using the workspace admin settings
To add a service principal to a workspace using the workspace admin settings page, the workspace must be enabled for identity federation.
As a workspace admin, log in to the Azure Databricks workspace.
Click your username in the top bar of the Azure Databricks workspace and select Admin Settings.
On the Service principals tab, click Add service principal.
Select an existing service principal to assign to the workspace or add a new one.
To add a new service principal, click the drop-down arrow in the search box and then click + Add new service principal. Paste the Application (client) ID for the service principal and enter a display name.
Note
If your workspace is not enabled for identity federation, you cannot assign existing account service principals to your workspace.
Assign a service principal to a workspace using REST APIs
The REST APIs that you can use to assign service principals to workspaces depend on whether the workspace is enabled for identity federation:
- Workspace enabled for identity federation: Account and workspace admins can use the Workspace Assignment API to assign service principals to workspaces. See Workspace Assignment API.
- Workspace not enabled for identity federation: A workspace admin can use the workspace-level SCIM (ServicePrincipal) API to assign service principals to their workspaces. See the Service Principal (SCIM) API.
Assign workspace admin rights to a service principal
You can assign the workspace admin role using the account console, workspace admin settings page, or REST APIs.
Assign the workspace admin role to a service principal using the account console
To assign the workspace admin role using the account console, the workspace must be enabled for identity federation.
- As an account admin, log in to the account console.
- Click
Workspaces. - On the Permissions tab, find the service principal.
- Click the
kebab menu at the far right of the user row and select Edit. - Under Role, choose Admin.
- Click Save.
To remove the admin role from a workspace service principal, perform the same steps, but choose User under Role.
Assign the workspace admin role to a service principal using the workspace admin settings page
To assign the workspace admin role using the workspace admin settings page, do the following:
- As a workspace admin, log in to the Azure Databricks workspace.
- Click your username in the top bar of the Azure Databricks workspace and select Admin Settings.
- On the Groups tab, select the Admins group.
- Click Add users or service principals.
- Select the service principal and click Confirm.
To remove the workspace admin role from a service principal, remove the service principal from the admin group.
Assign the workspace admin role to a service principal using the REST APIs
The REST APIs that you can use to assign the workspace admin role depend on whether the workspace is enabled for identity federation:
- Workspace enabled for identity federation: An account admin can use the account-level Workspace Assignment API to assign or remove the workspace admin role. Either an account admin or workspace admin can use the workspace-level Workspace Assignment API to perform this task.
- Workspace not enabled for identity federation: A workspace admin can use the workspace-level SCIM (Groups) REST API to assign a service principal to the admin group or remove them from the group. See the Service Principals API.
Deactivate a service principal in your Azure Databricks workspace
Workspace admins can deactivate service principals in a Azure Databricks workspace. A deactivated service principal cannot access the workspace from Azure Databricks APIs, however all of the service principal’s permissions and workspace objects remain unchanged. When a service principal is deactivated:
- The service principal cannot authenticate to the workspaces from any method.
- Applications or scripts that use the tokens generated by the service principal can no longer access the Databricks API. The tokens remain but cannot be used to authenticate while a service principal is deactivated.
- Clusters owned by the service principal remain running.
- Scheduled jobs created by the service principal have to be assigned to a new owner to prevent them from failing.
When a service principal is reactivated, they can authenticate to the workspace with the same permissions. Azure Databricks recommends deactivating service principal instead of removing them since removing a service principal is a destructive action. See Remove a service principal from a workspace.
Workspace admins can change a service principal’s status to deactivate the service principal using the workspace-level SCIM (Service Principals) API. For example:
curl --netrc -X PATCH \
https://<databricks-instance>/api/2.0/preview/scim/v2/ServicePrincipals/<id> \
--header 'Content-type: application/scim+json' \
--data @update-sp.json \
| jq .
update-sp.json:
{
"schemas": [ "urn:ietf:params:scim:api:messages:2.0:PatchOp" ],
"Operations": [
{
"op": "replace",
"path": "active",
"value": [
{
"value": "false"
}
]
}
]
}
Remove a service principal from a workspace
Account admins can remove service principals to identity federated workspaces using the account console and the Workspace Assignment API.
Workspace admins can remove service principals in their workspaces using the workspace admin settings page, the Workspace Assignment API, and the workspace-level SCIM (ServicePrincipals) API.
Remove a service principal from a workspace using the account console
- As an account admin, log in to the account console.
- In the sidebar, click Workspaces.
- Click your workspace name.
- On the Permissions tab, find the service principal.
- Click the
kebab menu at the far right of the service principal row and select Remove. - In the confirmation dialog box, click Remove.
Remove a service principal from a workspace using the workspace admin settings page
- As a workspace admin, log in to the Azure Databricks workspace.
- Click your username in the top bar of the Azure Databricks workspace and select Admin Settings.
- On the Service principals tab, find the service principal and click the
at the far right of the user row. - Click Delete to confirm.
Remove a service principal from a workspace using REST APIs
The REST APIs that you can use to remove service principals from workspaces depend on whether the workspace is enabled for identity federation as follows:
- Workspace enabled for identity federation: Account and workspace admins can use the Workspace Assignment API to remove service principals to workspaces. See Workspace Assignment API.
- Workspace not enabled for identity federation: A workspace admin can use the workspace-level SCIM (ServicePrincipal) API to remove service principals from their workspaces. See the Service Principals API.
Manage roles on a service principal
Important
This feature is in Public Preview.
You can grant Azure Databricks users, service principals, and account groups roles to manage and use a service principal. There are two roles that you can grant on a service principal: Service Principal Manager and Service Principal User. Roles cannot be granted to workspace-local groups.
- Service Principal Manager allows you to manage roles on a service principal. The creator of a service principal becomes the service principal manager. Account admins are also service principal managers on all service principals in an account.
Note
If a service principal was created before Jun 13, 2023, then the creator of the service principal does not have the service principal manager role by default. Please ask an account admin to grant you the service principal manager role.
- Service Principal User allows you to run jobs as the service principal. The job will run with the identity of the service principal, instead of the identity of the job’s owner. For more information, see Run a job as a service principal.
Note
Service principal managers do not inherit the service principal user role. If you want to use the service principal to execute jobs, you need to explicitly assign yourself the service principal user role.
For information on how to grant roles, see Roles for managing service principals.
Manage personal access tokens for a service principal
To authenticate a service principal to APIs on Azure Databricks, an administrator can create an Azure AD access token on behalf of the service principal. The Azure AD access token can be used to call Databricks REST APIs. Create an Azure AD access token by following these instructions:
Gather the following information:
Parameter Description Tenant ID The Directory (tenant) ID for the application registered in Azure AD. Client ID The Application (client) ID for the application registered in Azure AD. Client secret The Value of the client secret for the application registered in Azure AD. Use the preceding information along with curl to get the Azure AD access token.
curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \ https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/token \ -d 'client_id=<client-id>' \ -d 'grant_type=client_credentials' \ -d 'scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default' \ -d 'client_secret=<client-secret>'Replace:
<tenant-id>with the registered application’s tenant ID.<client-id>with the registered application’s client ID.<client-secret>with the registered application’s client secret value.
Do not change the value of the
scopeparameter. It represents the programmatic ID for Azure Databricks (2ff814a6-3304-4ab8-85cb-cd0e6f879c1d) along with the default scope (/.default, URL-encoded as%2f.default).For example:
curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \ https://login.microsoftonline.com/a1bc2d34-5e67-8f89-01ab-c2345d6c78de/oauth2/v2.0/token \ -d 'client_id=12a34b56-789c-0d12-e3fa-b456789c0123' \ -d 'grant_type=client_credentials' \ -d 'scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default' \ -d 'client_secret=abc1D~Ef...2ghIJKlM3'The Azure AD access token is in the
access_tokenvalue within the output of the call.
For additional, detailed step-by-step instructions for creating access tokens for service principals, see Service principals for Azure Databricks automation.
Note
It’s not possible to create, list, or manage a token for a service principal from within the Azure Databricks UI.
Manage entitlements for a service principal
An entitlement is a property that allows a user, service principal, or group to interact with Azure Databricks in a specified way. Entitlements are assigned to users at the workspace level. The following table lists entitlements and the workspace UI and API property name that you use to manage each one. You can use the workspace admin settings page and workspace-level SCIM REST APIs to manage entitlements.
| Entitlement name (UI) | Entitlement name (API) | Default | Description |
|---|---|---|---|
| Workspace access | workspace-access |
Granted by default. | When granted to a user or service principal, they can access the Data Science & Engineering and Databricks Machine Learning persona-based environments. Can’t be removed from workspace admins. |
| Databricks SQL access | databricks-sql-access |
Granted by default. | When granted to a user or service principal, they can access Databricks SQL. |
| Allow unrestricted cluster creation | allow-cluster-create |
Not granted to users or service principals by default. | When granted to a user or service principal, they can create clusters. You can restrict access to existing clusters using cluster-level permissions. Can’t be removed from workspace admins. |
| Allow pool creation (not available via UI) | allow-instance-pool-create |
Can’t be granted to individual users or service principals. | When granted to a group, its members can create instance pools. Can’t be removed from workspace admins. |
To add or remove an entitlement for a service principal, use the Service Principals API.
Feedback
Submit and view feedback for