Use serverless SQL warehouses

This article explains how to use serverless SQL warehouses and how to enable them for your account.

With serverless compute, the compute layer exists in your Azure Databricks account rather than your Azure subscription. This gives users in your account instant access to fully managed and elastic compute resources. Serverless compute is supported for use with Databricks SQL. Serverless compute does not affect how Databricks Runtime clusters work with notebooks and jobs.

Serverless SQL warehouses do not have public IP addresses. For more architectural information, see Serverless compute.

Important

Tasks on this page require that you have owner or contributor permissions on the Azure Databricks workspace.

Requirements

Serverless quotas

Serverless quotas are a safety measure for serverless compute. Serverless quotas restrict how many serverless compute resources a customer can have at any given time. The quota is enforced at the regional level for all workspaces in your account. Quotas are enforced only for serverless SQL warehouses. See Serverless quotas.

Configure Azure storage firewalls, if applicable

If you use an Azure Storage firewall to protect access to Azure storage data sources, you must configure your firewall to allow access from the serverless compute nodes. See Configure Azure storage firewalls to allow access from serverless SQL warehouses.

Create a serverless SQL warehouse

If your workspace is enabled for Databricks SQL Serverless, new SQL warehouses are serverless by default. The workspace must be in a supported region for serverless SQL warehouses and on the Premium pricing tier. For other requirements, see Requirements.

You can also upgrade an existing pro or classic SQL warehouse to serverless.

If you use the SQL Warehouse API to create the warehouse, serverless is not the default. Databricks recommends always explicitly setting enable_serverless_compute to true and warehouse_type to pro.

Auto-restart of serverless SQL warehouses

A serverless SQL warehouse auto-restarts in the following conditions:

  • You attempt to run a query that uses a stopped warehouse.
  • A job assigned to a stopped warehouse is scheduled to run.
  • You connect to a stopped warehouse from a JDBC/ODBC interface.
  • You open a query in the SQL editor that is saved to a stopped warehouse.
  • A dashboard is opened that is saved with a dashboard-level warehouse assigned.

See Warehouse auto-restart.

Limitations

Serverless warehouses have the following limitations: