Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Missing Hive Processors In Nifi 1.19.1

avatar
New Contributor

Hello,

 

I downloaded the latest Nifi version. i was able to connect nifi to an oracle database and it works fine. However, i need to connect nifi to hive, my problem is i cant find any processors related to hive. There is no SelectHiveQL processor and no PutHiveQL processor or any processor related to hive at all. not even the controller services for hive is found.

 

Mosoa_2-1671151915031.png

 

 

Mosoa_1-1671151817127.png

 

Am I doing anything wrong? I just need to connect Nifi to hive and i can't find the processors responsible for this.

 

Please help.

 

Thanks in advance

1 ACCEPTED SOLUTION

avatar
Super Mentor

@Mosoa 
Before upgrading Apache NiFi you should read through the migration guide to include all releases between your current release and the release you are upgrading to.
https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance

The Hive nar was removed as of Apache NiFi 1.17.0, so I am guessing your previous version was at least 1.16 or older.

As far as downloading and adding additional nars to NiFi, it is very easy.
1. Go to https://search.maven.org/ in your browser.
2. Search for Apache "NiFi Hive nar"
3. A list of artifacts will be shown, you'll need to click on all those you need, but I would start with "nifi-hive-nar" and "nifi-hive-services-api.nar" by clicking on the version number below each.
4. From the nar specific page you will see a "Downloads" option in the upper right corner of the page:

MattWho_1-1671209169860.png

5. When you click on it three option appear.  Select "nar".
6. Place the downloaded nar files in to the "lib" directory of your NiFi 1.19.1 installation. You'll notice that this directory already contains nar for other component classes already included with the base download. 

7. Make sure ownership and permissions on these new nar files match other nars in the "lib" directory.
8. Start your NiFi 1.19.1 

Now  you will see the Hive components available in your NiFi UI:

MattWho_2-1671209570106.png

 

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

View solution in original post

4 REPLIES 4

avatar

The hive libraries are quite large and are not included in the base binary distribution.    You can add them manually, or use github to clone and build the full project.

 

Reference: https://github.com/apache/nifi/blob/rel/nifi-1.19.0/nifi-assembly/pom.xml#L1054

 

 

avatar
New Contributor

Thank you for your reply,

I am quite new to this "adding a nifi library" thing, is there some steps that i need to follow ? i dont really understand and I've been searching for a guide to help me do this but i could not find any.

and i do not know what to do with the pom.xml 

 

any help will be apperciated.

 

Thanks.

avatar
Super Mentor

@Mosoa 
Before upgrading Apache NiFi you should read through the migration guide to include all releases between your current release and the release you are upgrading to.
https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance

The Hive nar was removed as of Apache NiFi 1.17.0, so I am guessing your previous version was at least 1.16 or older.

As far as downloading and adding additional nars to NiFi, it is very easy.
1. Go to https://search.maven.org/ in your browser.
2. Search for Apache "NiFi Hive nar"
3. A list of artifacts will be shown, you'll need to click on all those you need, but I would start with "nifi-hive-nar" and "nifi-hive-services-api.nar" by clicking on the version number below each.
4. From the nar specific page you will see a "Downloads" option in the upper right corner of the page:

MattWho_1-1671209169860.png

5. When you click on it three option appear.  Select "nar".
6. Place the downloaded nar files in to the "lib" directory of your NiFi 1.19.1 installation. You'll notice that this directory already contains nar for other component classes already included with the base download. 

7. Make sure ownership and permissions on these new nar files match other nars in the "lib" directory.
8. Start your NiFi 1.19.1 

Now  you will see the Hive components available in your NiFi UI:

MattWho_2-1671209570106.png

 

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

avatar
New Contributor

Thank you, man

 

I can now see hive processors

 

Mosoa_0-1671212367584.png