Support Questions
Find answers, ask questions, and share your expertise

NIFI 1.8.0 Error : Could not initialize class org.apache.hadoop.hive.conf.HiveConf

NIFI 1.8.0 Error : Could not initialize class org.apache.hadoop.hive.conf.HiveConf

New Contributor

Hi Team,


We are upgrading Apache NIFI version to 1.8.0 from 1.5.0. We are working with HDF 3.3.0. We are on a Cloudera 5.15 Cluster

We are using custom package called HiveConnectionPool which is bundled with hadoop version 2.7.3 and hive jdbc version 1.1.0. and we are getting below error while the processor is being initialized:


Error:

2019-05-22 07:00:13,797 ERROR [Validate Components Thread-4] o.a.nifi.dbcp.hive.HiveConnectionPool HiveConnectionPool[id=eb3d6a71-4802-3eb5-617c-f14a9d674ba3] Failed to perform validation due to java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hive.conf.HiveConf: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hive.conf.HiveConf

java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hive.conf.HiveConf

at org.apache.nifi.util.hive.HiveJdbcCommon.getConfigurationFromFiles(HiveJdbcCommon.java:379)

at org.apache.nifi.dbcp.hive.HiveConnectionPool.customValidate(HiveConnectionPool.java:173)

at org.apache.nifi.components.AbstractConfigurableComponent.validate(AbstractConfigurableComponent.java:126)

at org.apache.nifi.controller.AbstractComponentNode.computeValidationErrors(AbstractComponentNode.java:399)

at org.apache.nifi.controller.AbstractComponentNode.performValidation(AbstractComponentNode.java:382)

at org.apache.nifi.components.validation.StandardValidationTrigger.trigger(StandardValidationTrigger.java:52)

at org.apache.nifi.components.validation.TriggerValidationTask.run(TriggerValidationTask.java:42)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)2019-05-22 07:00:13,797 ERROR [Validate Components Thread-4] o.a.nifi.dbcp.hive.HiveConnectionPool HiveConnectionPool[id=eb3d6a71-4802-3eb5-617c-f14a9d674ba3] Failed to perform validation due to java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hive.conf.HiveConf: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hive.conf.HiveConf

java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hive.conf.HiveConf

at org.apache.nifi.util.hive.HiveJdbcCommon.getConfigurationFromFiles(HiveJdbcCommon.java:379)

at org.apache.nifi.dbcp.hive.HiveConnectionPool.customValidate(HiveConnectionPool.java:173)

at org.apache.nifi.components.AbstractConfigurableComponent.validate(AbstractConfigurableComponent.java:126)

at org.apache.nifi.controller.AbstractComponentNode.computeValidationErrors(AbstractComponentNode.java:399)

at org.apache.nifi.controller.AbstractComponentNode.performValidation(AbstractComponentNode.java:382)

at org.apache.nifi.components.validation.StandardValidationTrigger.trigger(StandardValidationTrigger.java:52)

at org.apache.nifi.components.validation.TriggerValidationTask.run(TriggerValidationTask.java:42)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

We have removed below default hive nars as we are using our custom nar:

nifi-hive-nar-1.8.0.3.3.1.0-10.nar

nifi-hive3-nar-1.8.0.3.3.1.0-10.nar


JAR/Version details:

Apache NIFI 1.5.0. is bundled with hadoop version 2.7.3 and hive jdbc version 1.1.0.

Apache NIFI 1.8.0. is bundled with hadoop version 3.0.0 and hive jdbc version 3.1.0.

We are using hive server version: Hive 1.1.0-cdh5.15.1


If we are using HiveConnectionPool provided by NIFI version 1.8.0 then we are getting client_protocol is unset error.

This indicates a version mismatch between client and server, namely that the client is newer than the server. So we cannot use default one

https://stackoverflow.com/questions/24694415/required-field-client-protocol-is-unset


Can you please suggest what else we can try to resolve this issue?