Serverless compute
With the serverless compute version of the Databricks platform architecture, the compute layer exists in your Azure Databricks account rather than your Azure subscription.
Databricks SQL Serverless
Databricks SQL Serverless supports serverless compute. Workspace admins can create serverless SQL warehouses that enable instant compute and are managed by Azure Databricks. Serverless SQL warehouses use compute clusters in your Azure Databricks account. Use them with Databricks SQL queries just like you normally would with customer-hosted SQL warehouses (classic and pro SQL warehouses).
Before you can create serverless SQL warehouses, your workspace must be in a supported region for serverless SQL warehouses, be on the Premium pricing tier, and must not use an external Hive metastore. Review additional limitations.
If your workspace meets those criteria:
- New SQL warehouses are serverless by default when created from the UI or the SQL Warehouses API, but you can also create new pro or classic SQL warehouses.
- You can upgrade a pro or classic SQL warehouse to a serverless SQL warehouse or a classic SQL warehouse to a pro SQL warehouse. You can also downgrade from serverless to pro or classic.
Use of the serverless SQL warehouses feature affects only the use with Databricks SQL. Databricks Runtime clusters continue to work with notebooks and jobs in the Data Science & Engineering or Databricks Machine Learning workspace environments. Databricks Runtime clusters always run in the classic compute plane in your Azure account. See Compare serverless compute to other Azure Databricks architectures.
Model Serving
Azure Databricks Model Serving deploys your MLflow machine learning (ML) models and exposes them as REST API endpoints that run in your Azure Databricks account. The serverless compute resources run as Azure Databricks Azure resources in what is known as the serverless compute plane.
In contrast, the legacy model serving architecture is a single-node cluster that runs in your Azure subscription within the classic compute plane.
- Easy configuration and compute resource management: Azure Databricks automatically prepares a production-ready environment for your model and makes it easy to switch its compute configuration.
- High availability and scalability: Serverless model endpoints autoscale, which means that the number of server replicas automatically adjusts based on the volume of scoring requests.
- Dashboards: Use the built-in serverless model endpoint dashboard to monitor the health of your model endpoints using metrics such as queries-per-second (QPS), latency, and error rate.
For regional support, see Azure Databricks regions.
Serverless quotas
Serverless quotas are a safety measure for serverless compute. Serverless quotas restrict how many serverless compute resources a customer can have at any given time. The quota is enforced at the regional level for all workspaces in your account. Quotas are enforced only for serverless SQL warehouses. See Serverless quotas.
Compare serverless compute to other Azure Databricks architectures
Azure Databricks operates out of a control plane and a compute plane:
- The control plane includes the backend services that Azure Databricks manages in its own Azure subscription. Databricks SQL queries, notebook commands, and many other workspace configurations are stored in the control plane and encrypted at rest.
- The compute plane is where data is processed by clusters of compute resources.
There are important differences between the classic compute plane (the original Azure Databricks platform architecture) and the serverless compute plane:
- For a classic compute plane, Azure Databricks compute resources run in your Azure subscription. Clusters perform distributed data analysis using queries (in Databricks SQL) or notebooks (in the Data Science & Engineering or Databricks Machine Learning environments):
- New clusters are created within each workspace’s virtual network in the customer’s Azure subscription.
- A classic compute plane has natural isolation because it runs in each customer’s own Azure subscription.
- For a serverless compute plane, Azure Databricks compute resources run in a compute layer within your Azure Databricks account:
- The serverless compute plane is used for serverless SQL warehouses and Model Serving. Enabling serverless compute does not change how Databricks Runtime clusters work in the Data Science & Engineering or Databricks Machine Learning environments.
- To protect customer data within the serverless compute plane, serverless compute runs within a network boundary for the workspace, with various layers of security to isolate different Azure Databricks customer workspaces and additional network controls between clusters of the same customer.
Azure Databricks creates a serverless compute plane in the same Azure region as your workspace’s classic compute plane.
The following diagram shows important differences between the serverless compute plane and classic compute plane for both serverless features.


For more information about secure cluster connectivity, which is mentioned in the diagram, see Secure cluster connectivity (No Public IP / NPIP).
The table below summarizes differences between serverless compute and the classic compute plane architecture of Azure Databricks, focusing on product security. It is not a complete explanation of those security features or a detailed comparison.
| Item | Serverless compute plane for SQL warehouses on Azure Databricks | Classic compute plane on Azure Databricks |
|---|---|---|
| Virtual private network for compute plane | The VNet is in the customer’s Azure Databricks account, with network boundaries between clusters. | The VNet is in the customer’s Azure subscription |
| Customize virtual network and firewall settings | No. | Yes, if you use VNet injection |
| Customize CIDR ranges | No. | Yes, if you use VNet injection |
| Container-level network isolation for clusters and SQL warehouses | Databricks managed software firewall. | Databricks managed software firewall |
| Communication between control plane and compute plane | Direct Databricks-managed mTLS encrypted communication, with connection initiated from the control plane. | When secure cluster connectivity (SCC) is enabled, which is the default for Azure, individual VMs connect to the SCC relay in the control plane during cluster creation. When SCC is disabled, the control plane connects to individual VMs using public IPs. |
Feedback
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback.
Submit and view feedback for