I can't run putHiveQL

[#1]

sorry experts if I'm asking a naive question but I'm new in using NiFi ,

,anyway,

I'm constructing a flow in which I'm getting data from a website and and put it in a hive table , but while I'm setting up the PutHiveQL processor I ran into an situation after I configure the service controller in the NiFi

put the putHiveQL processor always complain from HiveConnectionPool

within the Connection pool setting the following screenshot appears the complaining about the Database Connection URL

and Hbase client as well .

I think that I 've missed something I can't hit .

can anyone help me ?

thanks in advance .

[#2 · Accepted solution]

The HiveConnectionPool controller service needs to be configured to connect to a Hive instance. If you edit the controller service (using the pencil icon on the right side of the controller service entry in your Process Group Configuration window above, it will list the properties (required properties are in bold). The Database Connection URL is probably something like jdbc:hive2://sandbox.hortonworks.com:10000/default. The Hive Configuration Resources property should point at a comma-separated list of configuration files (usually core-site.xml and hive-site.xml at least). If you have username/password authentication set up for your Hive instance, you will need to supply values for those properties as well. Alternatively if your Hive instance is secured with Kerberos, you will need to supply values for those properties.

Once your HiveConnectionPool has been configured correctly (and saved by hitting the Apply button), you will need to enable it by clicking the lightning bolt icon (also on the right side near the pencil icon) then the Enable button. Return to the canvas and either route the "success" and/or "failure" connections from PutHiveQL to some other processor, or auto-terminate the relationship(s) by opening the Configuration dialog for PutHiveQL and selecting the checkboxes for the relationship(s) under the "Auto-terminate relationships" section. At that point your PutHiveQL processor should be valid (you will see a red square icon in the upper-left corner of the processor instead of the yellow triangle).

[#3]

@Mahmoud Shash

There was a bug identified in the Controller service UI of HDF 2.1.3. This bug affected users ability to modify, enable, disable and delete controller services. The HDF 2.1.3 release was pulled down. This bug was addresses in HDF 2.1.4. If you upgrade to HDF 2.1.4 you will be able to successfully access the Controller services in the CS UI.

Thanks,

Matt

[#4]

Dear @Matt Burgess what a luck ,yea it worked after upgrading to hdf3.0

Thanks a lot man:)

Appreciated a lot

[#5]

thanks @Matt Burgess for your reply but I believe I've missed something

what I did so far :

from the NiFi controller settings

1- I changed the controller service name

2- and change the Properties as you mentioned the DB and else:

3-enabling it

* from the putHiveQL:

I can't find the HiveConnectionPool which I've just created .

even though when I select any one of them it complains about DB connectivity issue as per my last screenshot in my previous post .

[#6]

What version of NiFi/HDF are you using? Also, are all those HiveConnectionPool instances valid (meaning they are used in various parts of your flow, in process groups, etc.)? If not, try deleting the invalid ones. Once you configure and save your FPCluster-HiveConnectionPool, it should be available to the Hive processors, so I'm not sure why you're not seeing it.

[#7]

HDF 2.1.3

and strange I can't find a way to delete the created HiveConfigurationPool instances

[#8]

@Mahmoud Shash

HDF 2.1.3 is a bad release. You are running into the exact Controller Service UI bug that resulted in HDF 2.1.3 being pulled and replaced with HDF 2.1.4. You can upgrade you HDF 2.1.3 to HDF 2.1.4 to fix this issue. Then you will be able to enable, disable, configure, and delete the HiveConnectionPool Controller service.

Matt