Skip to main content
Venkatd
Venkatd
Silver ⭐️
July 26, 2022
Solved

Unable to integrate kafka and hbase into atlas 2.2.0

Hi Team,

we are trying to integrate kafka and hbase into atlas 2.2.0
we are not using embedded kafka and hbase , seperately installed in same machine.
kafka version : 2.13-2.5.0
atlas version : 2.2.0

/home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh --topic Maestrotopic3
>>>>> /home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh
>>>>> /home/ranger/apache-atlas-sources-2.2.0
Using Kafka configuration directory [/data/kafka_2.13-2.5.0/config]
Log file for import is /var/log/atlas/import-kafka.log
Picked up _JAVA_OPTIONS: -Xms1024m -Xmx1024m -XX:MaxPermSize=512m
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.clas
s]
SLF4J: Found binding in [jar:file:/data/kafka_2.13-2.5.0/libs/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/exception/NestableRuntimeException
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at org.apache.atlas.kafka.bridge.KafkaBridge.main(KafkaBridge.java:95)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.exception.NestableRuntimeException
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 25 more
Failed to import Kafka Data Model!!!

FYI:
kafka-console-producer.sh and kafka-console-consumer.sh are running fine.
if we run the import-kafka.sh the without topic name , received similar error.

Can you please help .

 

 

Babasaheb
BabasahebAnswer
Gold ⭐️
August 7, 2022

@Venkatd 

 

As per the initial description, it looks like you have installed the required components manually and trying integrate them but you are getting the following error:

 

ERROR:

Exception in thread "main" java.lang.NoClassDefFoundError

 

Kindly note that the above exception usually occurs when the dependencies in the classpath are missing that are required for the script/component.

 

First thing please very kafka-client jar (for example kafka-client-2.13-2.5.0.jar) exists in the classpath. 

 

If yes, then it requires "commons-lang" jar/dependencies in your classpath.

 

If you found this response assisted with your query, please take a moment to log in and click on  KUDOS 🙂 & ”Accept as Solution" below this post.

 

Thank you.

View in thread

3 replies

Babasaheb
Gold ⭐️
BabasahebAnswer
Gold ⭐️
August 7, 2022

@Venkatd 

 

As per the initial description, it looks like you have installed the required components manually and trying integrate them but you are getting the following error:

 

ERROR:

Exception in thread "main" java.lang.NoClassDefFoundError

 

Kindly note that the above exception usually occurs when the dependencies in the classpath are missing that are required for the script/component.

 

First thing please very kafka-client jar (for example kafka-client-2.13-2.5.0.jar) exists in the classpath. 

 

If yes, then it requires "commons-lang" jar/dependencies in your classpath.

 

If you found this response assisted with your query, please take a moment to log in and click on  KUDOS 🙂 & ”Accept as Solution" below this post.

 

Thank you.

Translate
Venkatd
VenkatdAuthor
Silver ⭐️
August 23, 2022

Hi  @Babasaheb

 

Sorry I was in vacation , I could not reply. Thanks for your help , it has resolved the issue with kafka.

I am facing issue with hbase integration , raised request in below link.

 

Unable-integrate-hbase-with-Atlas/td-p/350587

Translate
VidyaSargur
Community Manager
VidyaSargur
Community Manager
August 17, 2022

@Venkatd, Has the reply helped resolve your issue? If so, can you please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future? 

Translate
Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
Terms & ConditionsAccessibility statement