Support Questions
Find answers, ask questions, and share your expertise

Kafka Handler for CDH 6.1.1 (Hive 2.1.1)

Solved Go to solution

Kafka Handler for CDH 6.1.1 (Hive 2.1.1)

New Contributor

We have CDH 6.1.1 cluster and Kafka cluster. I want to make possible to select data from Kafka topics using Hive. Is there any way to do it? Where I can find Kafka storage handler version for Hive 2.1.1 ?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Kafka Handler for CDH 6.1.1 (Hive 2.1.1)

Cloudera Employee

Hello @d33play 

I checked the documents again and found there is an internal jira CDH-72683 which was tracked for CDH versions to support Kafka Storage Handler. 
Kafka Storage Handler is a part of Hive 3.x version but CDH 6.1.1 comes with Apache Hive 2.1.1

Hence it is not supported in CDH versions but is available with CDP.

https://docs.cloudera.com/runtime/7.2.0/release-notes/topics/rt-runtime-component-versions.html

https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/runtime-release-notes/topics/rt-pvc-runtime-c...

 

So, the KafkaStorageHandler package will not work with Hive 2.1.1.

 

Hope this information helps.

 

Thanks and Regards,

Nandini

 

View solution in original post

3 REPLIES 3

Re: Kafka Handler for CDH 6.1.1 (Hive 2.1.1)

Cloudera Employee

I believe this is what you are looking for 

https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/integrating-hive-and-bi/topics/hive-kafka-int...

https://blog.cloudera.com/introducing-hive-kafka-sql/

 

And a more comprehensive doc - https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.5.1/kafka-hive-integration/hdf-kafka-hive-integrat...

 

You would not have to worry about the versions as the components are tested for cross compatibility. 

Re: Kafka Handler for CDH 6.1.1 (Hive 2.1.1)

New Contributor

Thank you for reply. Is there any installation guide (which jars, where to place etc.)?

I've tried ADD JAR from hive CLI (not beeline), then I trying to create kafka storage table it's returns error (Error: Error running query: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/DefaultHiveMetaHook).

Re: Kafka Handler for CDH 6.1.1 (Hive 2.1.1)

Cloudera Employee

Hello @d33play 

I checked the documents again and found there is an internal jira CDH-72683 which was tracked for CDH versions to support Kafka Storage Handler. 
Kafka Storage Handler is a part of Hive 3.x version but CDH 6.1.1 comes with Apache Hive 2.1.1

Hence it is not supported in CDH versions but is available with CDP.

https://docs.cloudera.com/runtime/7.2.0/release-notes/topics/rt-runtime-component-versions.html

https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/runtime-release-notes/topics/rt-pvc-runtime-c...

 

So, the KafkaStorageHandler package will not work with Hive 2.1.1.

 

Hope this information helps.

 

Thanks and Regards,

Nandini

 

View solution in original post