Hi,   I am actually using an Amazon EMR, which does not yet support hive 3.x in any version of EMR clusters, so this might make me drop this idea :(.   Is there any workaround? Perhaps something like a set of libs we can import in Hive 2.x so it supports HiveKafkaStorageHandler?   Thanks again! ... View more

Hi @Manohar Vanam, Thanks a lot for your quick answer. I was trying to test KafkaStorageHandler but I was not able to get it to work yet. I am having the following error: Exception in thread “main” java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/metadata/StorageHandlerInfo        at java.lang.Class.forName0(Native Method)        at java.lang.Class.forName(Class.java:348)        at org.apache.hadoop.hive.ql.parse.ParseUtils.ensureClassExists(ParseUtils.java:261)        at org.apache.hadoop.hive.ql.parse.StorageFormat.fillStorageFormat(StorageFormat.java:64)        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:11907)        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:11040)        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11153)        at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)        at java.lang.reflect.Method.invoke(Method.java:498)        at org.apache.hadoop.util.RunJar.run(RunJar.java:239)        at org.apache.hadoop.util.RunJar.main(RunJar.java:153) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.metadata.StorageHandlerInfo        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) I think this might be due to the version of Hive I am using. The setup I have uses Hive 2.3.x but apparently the lib for KafkaStorageHandler is for 3.1.x. Is this the issue? Have you had the same problem? Thanks! ... View more