question Recommended data quality test suite for Hive / Pig / Oozie in Archives of Support Questions (Read Only)

Recommended data quality test suite for Hive / Pig / Oozie

pbalasundaram — Fri, 16 Sep 2022 09:51:36 GMT

I am looking for Best practices around data quality Testing for hive / pig/ oozie based ETL.

Client is looking at tools like Data flux data quality for Hadoop .

If there are any alternate recommendations , please update this question.

nsabharwal — Tue, 08 Dec 2015 04:36:10 GMT

https://www.trifacta.com/ - It can be used for data quality checks during cleansing

http://vis.stanford.edu/wrangler/ - open source version of trifacta

pbalasundaram — Tue, 08 Dec 2015 04:49:42 GMT

Hi Neeraj - Trifacta seems to be a data wrangling tool, does it also provide data quality measures OOTB ?

nsabharwal — Tue, 08 Dec 2015 04:58:18 GMT

@pbalasundaram I have heard that it can be used for the quality while wrangling.

aervits — Wed, 03 Feb 2016 23:48:51 GMT

@pbalasundaram are you still having issues with this? Can you accept best answer or provide your own solution?

Eukrev — Wed, 25 May 2016 11:41:17 GMT

Hi Neeraj, for data quality testing is there a model script developed on pig or spark, rather than using a tool. Thanks.

KokaN — Mon, 07 Dec 2020 19:36:20 GMT

2020 Update, what are the preferred data quality tools compatible with CDH for Hive,Hbase and Solr? Our team is looking at Apache Griffin.

Regards,
Nithya Koka