Content
Databricks cluster failure : OutOfMemoryError
Getting a continous failures after a couple of days of stream runs with below error message, what could be potential permanent solution for the same ? Driver restarted - possibly due to an OutOfMemoryError - and this stream has been stopped. Please…
Tenant to Tenant cloning - Azure data components
Hello team, There is a requirement for us to clone Azure data components (Azure SQL DB, ADF, Azure Databricks, Azure Synapse Analytics) from one tenant to another tenant. There are a lot of resources that needs migration. Checking here to see if there…
Do we need two Azure databricks workspace to configure private endpoints
According to the Documentation for Enable Azure Private Link as a simplified deployment for Databricks, we need two workspace. One for the actual workload and another just for browser authentication. The Actual Workspace has databricks_ui_api endpoint…
Azure Databricks -Any pointers for cost optimization ?
Hello, We would like to do cost optimization for Azure Databricks resources. Is there any pointer/white paper/ reference article /blog post that might be useful ? Applicate your Response. Thank you!!
Configured privacy settings disallow access for workspace <workspace-id> over your current network.
Hi All, We have a couple of Databricks workspaces that are deployed with secure access - basically you can only log in when you are within the network, However, I am running into a problem where I sometimes get this error: “Configured privacy settings…
I can't scan Databricks Delta lake and Notebook when using Purview.
When using purview, only Databricks' Workspace and Hive metastores are scanned, but Notebooks and Delta Lake are not scanned. What could be the cause of this problem? What should I do to solve this problem?
multiple json files load through api
Hi Expert, How to load multiple json files using API URL in blob or databricks
How to fix Azure databricks Server Error: Cache is Stale
Good morning, I have been working for several months on azure databricks without any issue, using notebooks stored in databricks. This morning, while working with Databricks notebooks stored within the platform, I suddenly experienced a complete…
Bringing SAP data into Databricks
HI, I am wondering if there is a way to bring SAP data into databricks? Does databricks provides connector for this purpose? Regards Rajaniesh
Support of Multiselect List in Azure Data Factory parameters
Hello Team, We are working on a scenario where we want to provide a capability to user to select some values (Multiple) from a predefined list of values. And as a result, the data factory should run only for the selected values and not for all the values…
How to access Azure Devops from Databricks using a Service Principal
Hi. I am trying to install a package (Flask==2.0.2) from my Azure Devops portal using Databricks. For this purpose I am using a PAT and passing this in the %pip install statement in databricks. Now I have created a Service principal and used the same in…
How do we get the folder path of the file inside the workspace - Azure Databricks
I'm using the Databricks REST API to create the folder under the workspace, so that the folder will be accessed controlled as shown below URL - {{baseUrl}}/2.0/workspace/mkdirs Body - {"path":"/dde9540f/cert"} It's getting…
Azure databricks unity catalog service principal
I am trying to assign permissions (Account admin) to service principal in unity catalog . but it's appearing WITHOUT name in Unity catalog--> User management . Only id and status is showing. name is blank. I am not able to edit or delete this from…
Reading avro file in Databricks
How to read an .avro files stored in data lake using databricks.
suspecting wrong info by azure
In the document page ,it mentioned i can access databrciks in community https://docs.databricks.com/getting-started/community-edition.html, This article describes how to sign up for Databricks Community Edition. Unlike the Databricks Free Trial,…
Transition Azure Data Factory Mapping Data Flows to Azure Databricks (Scala?)
As a result of some infrastructure changes that are outside of my control, my organization is shifting from the use of Azure Data Factory (including Mapping Data Flows) to Azure Databricks. It is my understanding that ADF Mapping Data Flows functionally…
CNAME records for westus.pl-auth.azuredatabricks.net for multiple Azure databricks workspace usign custom DNS
How to set up CNAME records for multiple Azure Databricks workspaces for Azure Active Directory OAuth flow reply URL, using Custom DNS and my databricks workspaces has backend private endpoint configured but not browser authentication private…
azcopy upload file failed with error 403 using service principal
when i run azcopy upload files to storage account(subscription A) failed with error:403 using service principal from azure databrick's VM(subscription B, vent B, subnet B). i has granted storage blob contributor role to that service principal, also add…
I am trying to connect using databricks sql but I am getting error using node js
I am trying to connect using databricks but I am getting error. Pls help to resolved it. Here is Error Screenshot. Here is Code Screenshot
How to read append blob in (Azure Blob or ADLS) from Synapse or Databricks
Hi All I am trying to read a telemetry log , which is stored currently in blob storage without the hierarchical namespace enabled , when I try to read , I am getting the following error. All my files in storage account are currently saved as append blob,…