Member since
10-16-2018
2
Posts
0
Kudos Received
0
Solutions
10-16-2018
05:28 PM
Thanks for the response. Super helpful! Do you have any documentation or an example I can leverage to try to recreate some automated process to handle data drift?
... View more
10-16-2018
01:30 PM
I can't find a way to use Nifi to handle Data Drift with tables in Hive. I've used Streamsets at other companies and within their pipelines, Streamsets can manage the Hive Metadata for the tables created from the raw data. For example, if I am ETLing data from a flat file and that file has three fields: Name, ID, DOB and a fourth gets added, FavoriteColor, Streamsets will automatically add that column to the table in Hive I laid on-top of the data during the ETL process. In Nifi, it appears I have to manually change the table. Any ideas on how to do that same thing in Nifi?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache NiFi