Unable to integrate kafka and hbase into atlas 2.2.0

Venkatd — Tue, 26 Jul 2022 08:49:48 GMT

Hi Team,

we are trying to integrate kafka and hbase into atlas 2.2.0
we are not using embedded kafka and hbase , seperately installed in same machine.
kafka version : 2.13-2.5.0
atlas version : 2.2.0

/home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh --topic Maestrotopic3
>>>>> /home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh
>>>>> /home/ranger/apache-atlas-sources-2.2.0
Using Kafka configuration directory [/data/kafka_2.13-2.5.0/config]
Log file for import is /var/log/atlas/import-kafka.log
Picked up _JAVA_OPTIONS: -Xms1024m -Xmx1024m -XX:MaxPermSize=512m
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.clas
s]
SLF4J: Found binding in [jar:file:/data/kafka_2.13-2.5.0/libs/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/exception/NestableRuntimeException
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at org.apache.atlas.kafka.bridge.KafkaBridge.main(KafkaBridge.java:95)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.exception.NestableRuntimeException
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 25 more
Failed to import Kafka Data Model!!!

FYI:
kafka-console-producer.sh and kafka-console-consumer.sh are running fine.
if we run the import-kafka.sh the without topic name , received similar error.

Can you please help .

Re: Unable to integrate kafka and hbase into atlas 2.2.0

Babasaheb — Sun, 07 Aug 2022 06:41:03 GMT

@Venkatd

As per the initial description, it looks like you have installed the required components manually and trying integrate them but you are getting the following error:

ERROR:

Exception in thread "main" java.lang.NoClassDefFoundError

Kindly note that the above exception usually occurs when the dependencies in the classpath are missing that are required for the script/component.

First thing please very kafka-client jar (for example kafka-client-2.13-2.5.0.jar) exists in the classpath.

If yes, then it requires "commons-lang" jar/dependencies in your classpath.

If you found this response assisted with your query, please take a moment to log in and click on KUDOS 🙂 & ”Accept as Solution" below this post.

Thank you.

Re: Unable to integrate kafka and hbase into atlas 2.2.0

VidyaSargur — Wed, 17 Aug 2022 05:59:35 GMT

@Venkatd, Has the reply helped resolve your issue? If so, can you please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future?

Re: Unable to integrate kafka and hbase into atlas 2.2.0

Venkatd — Tue, 23 Aug 2022 05:54:23 GMT

Hi Babasaheb,

Sorry I was in vacation , I could not reply. Thanks for your help , it has resolved the issue with kafka.

I am facing issue with hbase integration , raised request in below link.

Unable-integrate-hbase-with-Atlas/td-p/350587

question Unable to integrate kafka and hbase into atlas 2.2.0 in Support Questions

Unable to integrate kafka and hbase into atlas 2.2.0

Re: Unable to integrate kafka and hbase into atlas 2.2.0

Re: Unable to integrate kafka and hbase into atlas 2.2.0

Re: Unable to integrate kafka and hbase into atlas 2.2.0