Kafka Handler for CDH 6.1.1 (Hive 2.1.1)

[#1]

We have CDH 6.1.1 cluster and Kafka cluster. I want to make possible to select data from Kafka topics using Hive. Is there any way to do it? Where I can find Kafka storage handler version for Hive 2.1.1 ?

[#2]

I believe this is what you are looking for 

https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/integrating-hive-and-bi/topics/hive-kafka-int...

https://blog.cloudera.com/introducing-hive-kafka-sql/

And a more comprehensive doc - https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.5.1/kafka-hive-integration/hdf-kafka-hive-integrat...

You would not have to worry about the versions as the components are tested for cross compatibility. 

[#3]

Thank you for reply. Is there any installation guide (which jars, where to place etc.)?

I've tried ADD JAR from hive CLI (not beeline), then I trying to create kafka storage table it's returns error (Error: Error running query: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/DefaultHiveMetaHook).

[#4 · Accepted solution]

Hello @d33play 

I checked the documents again and found there is an internal jira CDH-72683 which was tracked for CDH versions to support Kafka Storage Handler. 
Kafka Storage Handler is a part of Hive 3.x version but CDH 6.1.1 comes with Apache Hive 2.1.1

Hence it is not supported in CDH versions but is available with CDP.

https://docs.cloudera.com/runtime/7.2.0/release-notes/topics/rt-runtime-component-versions.html

https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/runtime-release-notes/topics/rt-pvc-runtime-c...

So, the KafkaStorageHandler package will not work with Hive 2.1.1.

Hope this information helps.

Thanks and Regards,

Nandini