anonymization

Presidio leverages ML models which might detect an instance in one sentence but not in another.
By automatically adding all instances of a previously identified entity, we can increase detection recall (and potentially decrease precision)

Example (hypothetical):

Text: "TrueForce is an American company. Lately, TrueForce was acquired by FalseForce".

If the first "TrueForce" was detected

Currently each Observer support state management.

Obsei store state in any of sqlalchemy supported database on schema mentioned in WorkflowTable class.

There is example also exist to show it's capability.

State m

There will be different options we can do in here:

• Trucation
• Geo IP to country
• Hashing

References:

• https://security.stackexchange.com/questions/18123/how-to-make-database-of-ip-addresses-anonymous/18178#18178
• https://tools.ietf.org/html/rfc6235
• https://github.com/oschwald/geoip2-golang

The following 3 attribute options (Month, Year, Days) are proposed.

1. StudyMonth: (example, 2012 February) 201202. A for loop of 31.
2. StudyYear: (example, 2013). 2013. A for loop of 31 * 12.
   The above 2 just assumes 31 days for all the months for simplicity. A C-FIND for 20120231 will return nothing. This is a quick and dirty approach. We should probably convert to date and handle better, e

Currently we support replacing usernames, emails, etc. I would like to expose the other data types the faker library supports as well.

In numpy typically columns are numbered using integers. Currently our implementation converts these into strings (i.e., '0', '1', etc.) in the returned generalizations. This is due to the bahavior of scikitlearn's OneHotEncoder which differs between integers and strings. But it would be better to leave these as integer keys to be consistent with numpy.
One workaround is to store in the beginning