pii

In the existing sample for batch analysis, the logic goes over all the columns and looks for PII. We'd like to extend this with logic which also evaluates how likely a column contains PII based on its name.

For example, if a column name is Age, and the values are [19,55,2,39], the column name coul

It is not surprising that deep and shallow scan show different results. Shallow scan only looks at column names. Deep scan looks at a sample of the data. I've even noticed that two different runs of deep scan show different results as sample rows are different. This is the challenge with not scanning all of the data. Its a trade-off between performance/cost and accuracy. There is no right answer.