Thanks a lot @TimothySpann for your time and insight. Your advice saves me from future futile efforts to hack/mess up with guava jar libs. I will probably try different approach to load Kafka topics into Hive. Thanks again.
... View more
Hello all, I tried to search Hive related topics but did not find the issue I face so I wanted to ask here. This is CDH 6.3.2 (Parcels) on Linux (CentOS) I have Kafka and Hive and I am able to create external Kafka tables after putting hive-exec-3.1.3000.7.1.3.0-100.jar and kafka-handler-3.1.3000.7.1.3.0-100.jar in Hive under Hive Auxiliary JARs Directory However, when I try to select data from the external table I get this error: 0: jdbc:hive2://localhost:10000/testdb> select * from kafka_table;
Error: java.io.IOException: java.io.IOException: java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;JJ)V (state=,code=0) After looking for this error I've found that it is most likely related to issues with different versions of guava in Hadoop and Hive: https://issues.apache.org/jira/browse/HIVE-22915 I see that Hadoop is using /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hadoop/client/guava.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hadoop-hdfs/lib/guava-11.0.2.jar And Hive is using /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hive/lib/guava-14.0.2.jar And Kafka is using /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/kafka/libs/guava-20.0.jar However following the advice from the HIVE Jira above if I equalize Hive and Hadoop versions (either downgrade Hive to guava-11.0.2.jar or upgrade Hadoop in all locations above to guava-14.0.2.jar) the errors persist. If I upgrade Hive to guava-20.0.jar I get multiple issues and I am not even able to read local tables: 0: jdbc:hive2://localhost:10000/testdb> desc kafka_table;
Error: Error running query: java.lang.IllegalAccessError: tried to access method com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator; from class org.apache.hadoop.hive.ql.exec.FetchOperator (state=,code=0) If I restore Hive to either guava-11.0.2.jar or guava-14.0.2.jar I am again able to select data from tables but initial error is still in place and I cannot read Kafka topic messages in Hive. Has anyone else faced this already and if yes what is recommended way to resolve this in Cloudera environment? Thanks a lot in advance. Warm regards, Yavor
... View more