Support Questions
Find answers, ask questions, and share your expertise

execute hive alter partition statement using nifi

Highlighted

execute hive alter partition statement using nifi

Explorer

I am new to the nifi, I need to execute the alter partition hive statement using nifi process. can anyone help on this.

Thanks in advance

5 REPLIES 5
Highlighted

Re: execute hive alter partition statement using nifi

Super Guru

There are many ways, I like the idea of using

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ExecuteStreamC...

And have your Hive DDL SQL in a script and a Shell script to run it. Whatever process you are doing now without NiFi, add some devops automation to it with NiFi. Is this something you need to only run once? Is this part of a load?

Re: execute hive alter partition statement using nifi

Explorer

Thanks for the reply, it need to run for no. of files, like suppose if we have 10 files this alter statement should execute 10 times.

Highlighted

Re: execute hive alter partition statement using nifi

Super Guru

In NiFi 0.7 and later (and in HDF 2.0), you can use the PutHiveQL processor to execute the Hive DDL. For your use case with multiple files, you could use ListFiles to get one flow file per filename. Then if you need the contents (i.e. if each file contains an ALTER statement), you can use FetchFile to get the contents and send that to PutHiveQL. If you just need some information about the file and will build an ALTER statement from it, you can use ListFile -> UpdateAttribute (to create an attribute with the information you need, such as part of the filename, using NiFi Expression Language) -> ReplaceText (to set the content -- also using Expression Language -- to the ALTER statement), then send that to PutHiveQL.

Highlighted

Re: execute hive alter partition statement using nifi

Explorer

Unfortunatlly I am using Apache Nifi 0.6

Highlighted

Re: execute hive alter partition statement using nifi

Explorer

Any help on this please