Created 12-15-2022 04:55 PM
Hello,
I downloaded the latest Nifi version. i was able to connect nifi to an oracle database and it works fine. However, i need to connect nifi to hive, my problem is i cant find any processors related to hive. There is no SelectHiveQL processor and no PutHiveQL processor or any processor related to hive at all. not even the controller services for hive is found.
Am I doing anything wrong? I just need to connect Nifi to hive and i can't find the processors responsible for this.
Please help.
Thanks in advance
Created 12-16-2022 08:53 AM
@Mosoa
Before upgrading Apache NiFi you should read through the migration guide to include all releases between your current release and the release you are upgrading to.
https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
The Hive nar was removed as of Apache NiFi 1.17.0, so I am guessing your previous version was at least 1.16 or older.
As far as downloading and adding additional nars to NiFi, it is very easy.
1. Go to https://search.maven.org/ in your browser.
2. Search for Apache "NiFi Hive nar"
3. A list of artifacts will be shown, you'll need to click on all those you need, but I would start with "nifi-hive-nar" and "nifi-hive-services-api.nar" by clicking on the version number below each.
4. From the nar specific page you will see a "Downloads" option in the upper right corner of the page:
5. When you click on it three option appear. Select "nar".
6. Place the downloaded nar files in to the "lib" directory of your NiFi 1.19.1 installation. You'll notice that this directory already contains nar for other component classes already included with the base download.
7. Make sure ownership and permissions on these new nar files match other nars in the "lib" directory.
8. Start your NiFi 1.19.1
Now you will see the Hive components available in your NiFi UI:
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt
Created 12-16-2022 05:19 AM
The hive libraries are quite large and are not included in the base binary distribution. You can add them manually, or use github to clone and build the full project.
Reference: https://github.com/apache/nifi/blob/rel/nifi-1.19.0/nifi-assembly/pom.xml#L1054
Created 12-16-2022 06:30 AM
Thank you for your reply,
I am quite new to this "adding a nifi library" thing, is there some steps that i need to follow ? i dont really understand and I've been searching for a guide to help me do this but i could not find any.
and i do not know what to do with the pom.xml
any help will be apperciated.
Thanks.
Created 12-16-2022 08:53 AM
@Mosoa
Before upgrading Apache NiFi you should read through the migration guide to include all releases between your current release and the release you are upgrading to.
https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
The Hive nar was removed as of Apache NiFi 1.17.0, so I am guessing your previous version was at least 1.16 or older.
As far as downloading and adding additional nars to NiFi, it is very easy.
1. Go to https://search.maven.org/ in your browser.
2. Search for Apache "NiFi Hive nar"
3. A list of artifacts will be shown, you'll need to click on all those you need, but I would start with "nifi-hive-nar" and "nifi-hive-services-api.nar" by clicking on the version number below each.
4. From the nar specific page you will see a "Downloads" option in the upper right corner of the page:
5. When you click on it three option appear. Select "nar".
6. Place the downloaded nar files in to the "lib" directory of your NiFi 1.19.1 installation. You'll notice that this directory already contains nar for other component classes already included with the base download.
7. Make sure ownership and permissions on these new nar files match other nars in the "lib" directory.
8. Start your NiFi 1.19.1
Now you will see the Hive components available in your NiFi UI:
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt
Created 12-16-2022 09:39 AM
Thank you, man
I can now see hive processors