Update Oracle Connection version 2.0 from 1.0
This is connected to another question I have. https://learn.microsoft.com/en-us/answers/questions/5490043/adf-oracle-connection-deprecated-version-1-0 Since the version 1 will be decommissioned soon. We need to update our Oracle connections to version 2.…
Azure Data Factory
Delete records from a salesforce object using Azure data factory
Hi there, I am working on a ADF pipeline that transforms data from salesforce and finally i need to delete some records from salesforce object. adf does not have a built in feature to delete and i was wondering if we can utilize lookup activity to run …
Azure Data Factory
Monitoring Shows Lower Inserted Row Count After Switching Azure SQL DB to UTF-8 Collation
Hello, I'm encountering an unusual monitoring issue after changing my Azure SQL Database collation to UTF-8 (e.g., Latin1_General_100_CI_AS_SC_UTF8). Here’s the scenario: I load 40679 records from a Parquet file (UTF-8 encoded data) into the database…
Azure Data Factory
ADF Logging Strategy – Expert Insights Wanted!
ADF Logging Strategy – Expert Insights Wanted! I'm currently building a reusable logging framework in Azure Data Factory (ADF) to capture execution metadata for key activities like Data Flows, Copy Activities, and Stored Procedures — and I’d love to get…
Azure Data Factory
Data Flow Sink with "Continue on Error" and Output to separate file not getting created
Hello, I have a Sink destination linked to Azure SQL DB in a data flow. Where I wanted to capture database rejected records into Azure BLOB file. I have tried choose to Continue on error with Transaction commit Single/Batch and selected Output to…
Azure Data Factory
Why are ADF Pipeline Parameters Not Appearing in Override Template Section?
I am trying to parameterize ADF Pipelines parameters in azure devops so that it can be overwritten before deployment to prod, I have added the below mentioned code in arm-template-parameters-definition.json file for the same but still I don't see the…
Azure Data Factory
How to set column name as the file name?
I've set the original file name as a column through the copy activity, but I can't get the data flow to set the column as the file name. Copy behavior is set to flatten hierarchy, then I use a data flow for some additional transformation. In the data…
Azure Data Factory
Azure Data Factory: Issues with Oracle Linked Service v2 in Copy Activity
We are working on updating Oracle Linked Services in Azure Data Factory (ADF) to version 2.0 and we have found several differences in behaviour compare with Oracle Connector v1.0: In Lookup activities, version 1.0 allowed semicolons at the end of…
Azure Data Factory
Does Microsoft perform regular penetration testing on their own systems
As per the heading, does Microsoft perform regular penetration testing on their own systems? I can't seem to find an answer to what I would have thought is a simple question and something they would surely advertise if they did?
Azure Data Factory
How to overcome ADF lookup activity limitation
Scenario I have a use case where data needs to be read from a source (SQL, Blob, S3, etc.), sent to a REST API for a custom transformation, and then the API response must be written back to a sink (SQL, S3, Blob, etc.). Current Approach Step 1: Use a…
Azure Data Factory
How to make the Chekpoint Key / SAP to stage subscriber process a static value?
Context: I am currently building a pipeline that extracts data from a SAP CDC source, where I want to utilize the delta mechanism this connector enables. My setup is: SAP CDC connector to source system (SAP CRM)- static subscriber name and ODP…
Azure Data Factory
Error_Message
Hello Team, I have setup the email Notification via logic app in ADF Pipeline.Please refer the screenshot below : But whenever the Pipeline get failed.In the email notification will not give the exact error.It will display only the pipeline got…
Azure Data Factory
Unable to download Purview exports
Hello I have tried to download the export packages from the ediscovery menu. I have tried on mac & linux Edge and firefox browsers. When I push the download button, nothing happens. The close button does work. Is there another way to access these…
Azure Data Factory
: Error during execution: Azure Data Factory Oracle connector 2.0 → Parquet write failure (ArrayIndexOutOfBounds: Index 126 out of bounds for length 38)
Hello — I'm hitting a blocking error when running an ADF pipeline that reads from Oracle (using Oracle connector 2.0) and writes to Parquet. I suspect it’s related to decimal/float precision mapping, but I can't find a definitive fix. Any guidance…
Azure Data Factory
How to use system variables in Azure data factory ?
I am new to ADF and need some help or guidance to use system variables in Azure data factory ? Thank you Himanshu
Azure Data Factory
The Data Factory Spark v2 linked service reports "The remote certificate is invalid" even though enableServerCertificateValidation is false.
Error message: System.Net.WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid…
Azure Data Factory
How to get the new data factory oracle 2.0 connector to connect?
We currently connect to oracle using the 1.0 connector. We must be moved to the new oracle 2.0 connector before end of support, 7/31/25. I was able to get this to connect to a non encrypted database, but unable to connect to an encrypted database. I had…
Azure Data Factory
PostgreSQL V2 Connector runs into an error when trying to copy timestamp with time zone values
We have an on-premises PostgreSQL (9.0.5) database from which I am staging data to an Azure SQL Database. So far, we have been using the PostgreSQL V1 Connector. I am currently trying to switch to V2, as V1 will be deprecated at the end of next…
Azure Data Factory
Unable to create new linked service for salesforce. As it changed from username password to Client ID and Client secret
Unable to establish connection to the Salesforce environment using provided client id and client secret, please verify the details. Error message: {"error":"invalid_grant","error_description":"no client credentials user…
Azure Data Factory
ADLS Archive Path (compared to Hyperscale)
In our project we have a set of archive tables from DB2 iSeries. These are static datasets (historical claims data from 2007–2019, frozen and never updated). They are not part of FlashCopy or CDC, but the customer wants them in a queryable form. One…