Serverless compute
Important
Serverless SQL warehouses are available in Public Preview. To start using Databricks SQL Serverless on Azure, please submit your request using this form.
With the Serverless compute version of the Databricks platform architecture, the compute layer exists in the Azure subscription of Azure Databricks rather than the customer’s Azure subscription.
As of the current release, Serverless compute is supported for use with Databricks SQL. Admins can create Serverless SQL warehouses (formerly SQL endpoints) that enable instant compute and are managed by Azure Databricks. Serverless SQL warehouses use compute clusters in the Azure subscription of Azure Databricks. Use them with Databricks SQL queries just like you normally would with the original customer-hosted SQL warehouses, which are now called Classic SQL warehouses.
Databricks changed the name from SQL endpoint to SQL warehouse because, in the industry, endpoint refers to either a remote computing device that communicates with a network that it’s connected to, or an entry point to a cloud service. A data warehouse is a data management system that stores current and historical data from multiple sources in a business friendly manner for easier insights and reporting. SQL warehouse accurately describes the full capabilities of this compute resource.
Before you can create Serverless SQL warehouses, you must enable Serverless Databricks SQL warehouses for your workspace. If Serverless SQL warehouses are enabled for your workspace:
- New SQL warehouses are Serverless by default when created from the UI or the API, but you can also create new Classic SQL warehouses.
- You can convert a Classic SQL warehouse to a Serverless SQL warehouse or convert from Serverless to Classic.
- This feature only affects Databricks SQL. It does not affect how Databricks Runtime clusters work with notebooks and jobs in the Data Science & Engineering or Databricks Machine Learning workspace environments. Databricks Runtime clusters always run in the Classic data plane in your Azure account. See Compare Serverless compute to other Databricks architectures.
Azure Databricks supports Serverless SQL warehouses in Azure region East US (eastus), East US 2 (eastus2), and West Europe (westeurope).
Compare Serverless compute to other Databricks architectures
Azure Databricks operates out of a control plane and a data plane:
- The control plane includes the backend services that Azure Databricks manages in its own Azure subscription. Databricks SQL queries, notebook commands, and many other workspace configurations are stored in the control plane and encrypted at rest.
- The data plane is where data is processed by clusters of compute resources.
There are important differences between the Classic data plane (the original Azure Databricks platform architecture) and the Serverless data plane:
- For a Classic data plane, Azure Databricks compute resources run in the customer’s Azure subscription. Clusters perform distributed data analysis using queries (in Databricks SQL) or notebooks (in the Data Science & Engineering or Databricks Machine Learning environments):
- New clusters are created within each workspace’s virtual network in the customer’s Azure subscription.
- A Classic data plane has natural isolation because it runs in each customer’s own Azure subscription. A Classic data plane is not a shared resource for multiple customers.
- For a Serverless data plane, Azure Databricks compute resources run in a special compute layer within the Azure subscription for Azure Databricks:
- As of this release, the Serverless data plane is used only for Serverless SQL warehouses. Enabling this feature does not change how Databricks Runtime clusters work in the Data Science & Engineering or Databricks Machine Learning environments.
- The Serverless data plane is a shared resource in the Azure subscription of Azure Databricks for multiple Azure Databricks customers.
- To protect customer data within the Serverless data plane, Serverless compute runs within a network boundary for the workspace, with various layers of security to isolate different Azure Databricks customer workspaces and additional network controls between clusters of the same customer.
Azure Databricks creates a Serverless data plane in the same Azure region as your workspace’s Classic data plane.
The following diagram shows important differences between the Serverless data plane and Classic data plane.

For more information about secure cluster connectivity, which is mentioned in the diagram, see Secure cluster connectivity (No Public IP / NPIP).
The table below summarizes differences between Serverless compute and the Classic data plane architecture of Azure Databricks, focusing on product security. It is not a complete explanation of those security features or a detailed comparison. For more details about Serverless compute security, or if you have questions about items in this table, contact your Azure Databricks representative.
| Item | Serverless data plane for SQL warehouses on Azure Databricks | Classic data plane on Azure Databricks |
|---|---|---|
| Virtual private network for data plane | The VNet in the Azure Databricks subscription is shared among customers, with additional network boundaries between clusters. | The VNet is in customer’s subscription |
| Customize virtual network and firewall settings | No. | Yes, if you use VNet injection |
| Customize CIDR ranges | No. | Yes, if you use VNet injection |
| Container-level network isolation for clusters and SQL warehouses | Databricks managed software firewall. | Databricks managed software firewall |
| Communication between control plane and data plane | Direct Databricks-managed TLS encrypted communication using private connectivity, with connection initiated from the control plane. | When secure cluster connectivity (SCC) is enabled, which is the default for Azure, individual VMs connect to the SCC relay in the control plane during cluster creation. When SCC is disabled, the control plane connects to individual VMs using public IPs. |
Feedback
Submit and view feedback for