Create HTML profiling reports from pandas DataFrame objects

📚 Papers by organizations sharing their work on applied data science & machine learning.

Data validation and organization of metadata for data frames and database tables

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

Profile and monitor your ML data pipeline end-to-end

Qualitis is a one-stop data quality management platform that supports quality verification, notification, and management for various datasource. It is used to solve various data quality problems caused by data processing. https://github.com/WeBankFinTech/Qualitis

Profile and monitor your ML data pipeline end-to-end

An RDF Unit Testing Suite

🐳 Tool to automate data quality checks on data pipelines

NBi is a testing framework (add-on to NUnit) for Business Intelligence and Data Access. The main goal of this framework is to let users create tests with a declarative approach based on an Xml syntax. By the means of NBi, you don't need to develop C# or Java code to specify your tests! Either, you don't need Visual Studio or Eclipse to compile your test suite. Just create an Xml file and let the framework interpret it and play your tests. The framework is designed as an add-on of NUnit but with the possibility to port it easily to other testing frameworks.

Jumbune, an open source BigData APM & Data Quality Management Platform for Data Clouds. Enterprise feature offering is available at http://jumbune.com. More details of open source offering are at,

数据治理、数据质量检核/监控平台（Django+jQuery+MySQL）

Librería para la evaluación de calidad de datos, e interacción con el portal de datos.gov.co

Lightweight library to write, orchestrate and test your SQL ETL. Writing ETL with data integrity in mind.

A tool to help improve data quality standards in observational data science.

A GitHub Action that makes it easy to use Great Expectations to validate your data pipelines in your CI workflows.

DTCleaner: data cleaning using multi-target decision trees.

Automated data quality suggestions and analysis with Deequ on AWS Glue

Migrated to: https://gitlab.com/Oslandia/osm-data-classification

DataOps for Government

Great Expectations Airflow operator

hive_compared_bq compares/validates 2 (SQL like) tables, and graphically shows the rows/columns that are different.

The PEDSnet Data Quality Assessment Toolkit (OMOP CDM)

QAMyData, a data quality assurance tool for SPSS, STATA, SAS and CSV files.

Data Quality Monitoring Tool

R package based on "Column Names as Contracts" blog post (https://emilyriederer.netlify.app/post/column-name-contracts/)

Implementation of Estimating Training Data Influence by Tracing Gradient Descent (NeurIPS 2020)

Data validation library for PySpark 3.0.0