We are working on HDF nifi cluster. We would like to have some workflow feature on HDF. e.g. If I can execute the oozie task after the data was imported to hdfs through nifi.
Is there any idea to implement the behavior.
Hello, @Artem Ervits
Thanks for your quickly response. There is my requirement:
1. To ETL Mysql data to HDFS, the data is increment data.
2. when the data ETL over then call an oozie job to merge data. and so on.
The problem is difficult to check the ETL is over, the increment data are insert to hdfs.
Could you give me advise?
For incremental loads from RDBMS, Nifi has QueryDatabaseTable processor, will track updates and inserts to tables but not deletes. If you can live with that limitation, you can get by without Oozie.
Yes, you're right.
However, I would like to know how to check the data flow is over?
It is must to check the status of every components, for me, It is difficult. Could you give me any points?