Support Questions

Find answers, ask questions, and share your expertise

Unable to integrate kafka and hbase into atlas 2.2.0

avatar
Explorer

Hi Team,

we are trying to integrate kafka and hbase into atlas 2.2.0
we are not using embedded kafka and hbase , seperately installed in same machine.
kafka version : 2.13-2.5.0
atlas version : 2.2.0

/home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh --topic Maestrotopic3
>>>>> /home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh
>>>>> /home/ranger/apache-atlas-sources-2.2.0
Using Kafka configuration directory [/data/kafka_2.13-2.5.0/config]
Log file for import is /var/log/atlas/import-kafka.log
Picked up _JAVA_OPTIONS: -Xms1024m -Xmx1024m -XX:MaxPermSize=512m
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.clas
s]
SLF4J: Found binding in [jar:file:/data/kafka_2.13-2.5.0/libs/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/exception/NestableRuntimeException
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at org.apache.atlas.kafka.bridge.KafkaBridge.main(KafkaBridge.java:95)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.exception.NestableRuntimeException
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 25 more
Failed to import Kafka Data Model!!!

FYI:
kafka-console-producer.sh and kafka-console-consumer.sh are running fine.
if we run the import-kafka.sh the without topic name , received similar error.

Can you please help .

 

 

2 ACCEPTED SOLUTIONS

avatar
Super Collaborator

@Venkatd 

 

As per the initial description, it looks like you have installed the required components manually and trying integrate them but you are getting the following error:

 

ERROR:

Exception in thread "main" java.lang.NoClassDefFoundError

 

Kindly note that the above exception usually occurs when the dependencies in the classpath are missing that are required for the script/component.

 

First thing please very kafka-client jar (for example kafka-client-2.13-2.5.0.jar) exists in the classpath. 

 

If yes, then it requires "commons-lang" jar/dependencies in your classpath.

 

If you found this response assisted with your query, please take a moment to log in and click on  KUDOS 🙂 & ”Accept as Solution" below this post.

 

Thank you.

View solution in original post

avatar
Explorer

Hi  Babasaheb,

 

Sorry I was in vacation , I could not reply. Thanks for your help , it has resolved the issue with kafka.

I am facing issue with hbase integration , raised request in below link.

 

Unable-integrate-hbase-with-Atlas/td-p/350587

View solution in original post

3 REPLIES 3

avatar
Super Collaborator

@Venkatd 

 

As per the initial description, it looks like you have installed the required components manually and trying integrate them but you are getting the following error:

 

ERROR:

Exception in thread "main" java.lang.NoClassDefFoundError

 

Kindly note that the above exception usually occurs when the dependencies in the classpath are missing that are required for the script/component.

 

First thing please very kafka-client jar (for example kafka-client-2.13-2.5.0.jar) exists in the classpath. 

 

If yes, then it requires "commons-lang" jar/dependencies in your classpath.

 

If you found this response assisted with your query, please take a moment to log in and click on  KUDOS 🙂 & ”Accept as Solution" below this post.

 

Thank you.

avatar
Explorer

Hi  Babasaheb,

 

Sorry I was in vacation , I could not reply. Thanks for your help , it has resolved the issue with kafka.

I am facing issue with hbase integration , raised request in below link.

 

Unable-integrate-hbase-with-Atlas/td-p/350587

avatar
Community Manager

@Venkatd, Has the reply helped resolve your issue? If so, can you please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future? 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: