Support Questions
Find answers, ask questions, and share your expertise

ConvertJsonToSQL: Connection with Hive DB

Rising Star

How do we connect a ConvertJsonToSQL processor to a Hive DB?

You cannot select a HiveConnectionPool as the value of the JDBC Connection Pool property and if i try to configure a DBCP connection pool with Hive like this:



I get a driver error: Cannot load JDBC driver class for the class i have configured.

My flow uses an incoming JSON file to create

the SQL statement with the ConvertJsonToSQL processor, and then i want to send the flowfile with the SQL statement to PutHiveQL to insert the data into Hive.

Is there something wrong with my approach?

Thanks in advance!


Re: ConvertJsonToSQL: Connection with Hive DB

Super Guru

HiveConnectionPool belongs to the NiFi Hive NAR, which includes a Hive JDBC driver. ConvertJSONToSQL is not in that NAR and thus does not have access to the driver. In this case you would want to download the driver (and any dependencies) into some folder (NOT NiFi's lib/ folder), and add the path to that folder as your value for "Database Driver Location(s)". Then the processor should be able to find the Hive driver.

Having said that, I can't remember offhand if this processor works with a Hive driver or not. If it does, you may still find that you have to use ReplaceText or something to change the generated SQL to proper HiveQL before executing it (presumably with PutHiveQL).