Created 09-14-2016 01:11 PM
I am new to the nifi, I need to execute the alter partition hive statement using nifi process. can anyone help on this.
Thanks in advance
Created 09-14-2016 01:18 PM
There are many ways, I like the idea of using
And have your Hive DDL SQL in a script and a Shell script to run it. Whatever process you are doing now without NiFi, add some devops automation to it with NiFi. Is this something you need to only run once? Is this part of a load?
Created 09-14-2016 04:18 PM
Thanks for the reply, it need to run for no. of files, like suppose if we have 10 files this alter statement should execute 10 times.
Created 09-14-2016 05:24 PM
In NiFi 0.7 and later (and in HDF 2.0), you can use the PutHiveQL processor to execute the Hive DDL. For your use case with multiple files, you could use ListFiles to get one flow file per filename. Then if you need the contents (i.e. if each file contains an ALTER statement), you can use FetchFile to get the contents and send that to PutHiveQL. If you just need some information about the file and will build an ALTER statement from it, you can use ListFile -> UpdateAttribute (to create an attribute with the information you need, such as part of the filename, using NiFi Expression Language) -> ReplaceText (to set the content -- also using Expression Language -- to the ALTER statement), then send that to PutHiveQL.
Created 09-16-2016 12:34 PM
Unfortunatlly I am using Apache Nifi 0.6
Created 09-19-2016 05:26 PM
Any help on this please