Member since
07-26-2022
6
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 429 | 08-22-2022 10:54 PM |
11-03-2022
12:26 AM
Hi Shelton, We have installed apache atlas 2.2.0 in VM , also installed required component like zookeeper, solr, hbase and kafka in same VM . we have ADLS storage account , we would like integrate ADLS to atlas .. Can you please help me on this ? Thanks, Venkat
... View more
09-29-2022
03:34 AM
Hi AlI, I need some help and idea in how do we integrate atlas with Azure ADLS. Can anyone provide high idea information. We have installed apache atlas on azure vm and are trying to load adls datasets and ADB processes into apache atlas .But unable to establish the connections .Need guidance .
... View more
Labels:
- Labels:
-
Apache Atlas
09-05-2022
07:13 AM
Hi All, We are enabling hbase hook in apache atlas 2.2.0 , we followed below steps , https://atlas.apache.org/2.0.0/Hook-HBase.html After that , when we start to start hbase , failing with below error. ROD. 2022-09-05 14:02:23,481 INFO [master/vmazewdranger01:16000:becomeActiveMaster] atlas.ApplicationProperties: Setting solr.wait-searcher property 'true' 2022-09-05 14:02:23,481 INFO [master/vmazewdranger01:16000:becomeActiveMaster] atlas.ApplicationProperties: Setting index.search.map-na me property 'false' 2022-09-05 14:02:23,482 INFO [master/vmazewdranger01:16000:becomeActiveMaster] hook.AtlasHook: Failed to load application properties org.apache.atlas.AtlasException: Failed to load application properties at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:150) at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:103) at org.apache.atlas.hook.AtlasHook.<clinit>(AtlasHook.java:80) at org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.<init>(HBaseAtlasCoprocessor.java:46) 2022-09-05 14:02:23,484 ERROR [master/vmazewdranger01:16000:becomeActiveMaster] coprocessor.CoprocessorHost: The coprocessor org.apache. atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: java.lang.InstantiationException: org.apache.atlas.hbase.hook.HBaseAtl asCoprocessor java.io.IOException: java.lang.InstantiationException: org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:280) at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:171) 2022-09-05 14:02:23,487 ERROR [master/vmazewdranger01:16000:becomeActiveMaster] master.HMaster: ***** ABORTING master vmazewdranger01,16 000,1662386539831: The coprocessor org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: java.lang.InstantiationE xception: org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor ***** java.io.IOException: java.lang.InstantiationException: org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor Thanks, Venkat
... View more
Labels:
- Labels:
-
Apache Atlas
-
Apache HBase
08-22-2022
10:54 PM
Hi Babasaheb, Sorry I was in vacation , I could not reply. Thanks for your help , it has resolved the issue with kafka. I am facing issue with hbase integration , raised request in below link. Unable-integrate-hbase-with-Atlas/td-p/350587
... View more
08-22-2022
10:50 PM
Unable integrate hbase with Atlas. we are trying to integrate hbase into atlas 2.2.0 we are not using embedded kafka and hbase , seperately installed in same machine. hbase version:2.4.13 ranger@:~/apache-atlas-sources-2.2.0/hook-bin$ ./import-hbase.sh -n <hbase namespace name> >>>>> ./import-hbase.sh >>>>> /home/ranger/apache-atlas-sources-2.2.0 Using HBase configuration directory [/data/hbase/conf/] Log file for import is /var/log/atlas/import-hbase.log Picked up _JAVA_OPTIONS: -Xms1024m -Xmx1024m -XX:MaxPermSize=512m OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0 ImportHBaseEntities failed. Please check the log file for the detailed error message Failed to import HBase Data Model!!! Output of /var/log/atlas/import-hbase.log: 2022-08-23 05:33:25,927 INFO - [main:] ~ Looking for atlas-application.properties in classpath (ApplicationProperties:124) 2022-08-23 05:33:25,931 INFO - [main:] ~ Loading atlas-application.properties from file:/data/hbase/conf/atlas-application.properties (ApplicationProperties:137) 2022-08-23 05:33:25,960 INFO - [main:] ~ Using graphdb backend 'janus' (ApplicationProperties:317) 2022-08-23 05:33:25,961 INFO - [main:] ~ Using storage backend 'hbase2' (ApplicationProperties:328) 2022-08-23 05:33:25,962 INFO - [main:] ~ Using index backend 'solr' (ApplicationProperties:339) 2022-08-23 05:33:25,962 INFO - [main:] ~ Atlas is running in MODE: PROD. (ApplicationProperties:343) 2022-08-23 05:33:25,963 INFO - [main:] ~ Setting solr.wait-searcher property 'true' (ApplicationProperties:350) 2022-08-23 05:33:25,963 INFO - [main:] ~ Setting index.search.map-name property 'false' (ApplicationProperties:354) 2022-08-23 05:33:25,963 INFO - [main:] ~ Setting atlas.graph.index.search.max-result-set-size = 150 (ApplicationProperties:364) 2022-08-23 05:33:25,963 INFO - [main:] ~ Setting atlas.graph.index.search.solr.wait-searcher = true (ApplicationProperties:365) 2022-08-23 05:33:25,963 INFO - [main:] ~ Property (set to default) atlas.graph.cache.db-cache = true (ApplicationProperties:377) 2022-08-23 05:33:25,964 INFO - [main:] ~ Property (set to default) atlas.graph.cache.db-cache-clean-wait = 20 (ApplicationProperties:377) 2022-08-23 05:33:25,964 INFO - [main:] ~ Property (set to default) atlas.graph.cache.db-cache-size = 0.5 (ApplicationProperties:377) 2022-08-23 05:33:25,964 INFO - [main:] ~ Property (set to default) atlas.graph.cache.tx-cache-size = 15000 (ApplicationProperties:377) 2022-08-23 05:33:25,964 INFO - [main:] ~ Property (set to default) atlas.graph.cache.tx-dirty-size = 120 (ApplicationProperties:377) 2022-08-23 05:33:25,971 ERROR - [main:] ~ ImportHBaseEntities failed (HBaseBridge:198) org.apache.atlas.AtlasException: Failed to load application properties at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:150) at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:103) at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:140) Caused by: org.apache.commons.configuration.ConversionException: 'atlas.graph.index.search.map-name' doesn't map to a List object: false, a java.lang.Boolean at org.apache.commons.configuration.AbstractConfiguration.getList(AbstractConfiguration.java:1144) at org.apache.commons.configuration.AbstractConfiguration.getList(AbstractConfiguration.java:1109) at org.apache.commons.configuration.AbstractConfiguration.interpolatedConfiguration(AbstractConfiguration.java:1274) at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:145) ... 2 more 2022-08-23 05:35:07,422 INFO - [main:] ~ Looking for atlas-application.properties in classpath (ApplicationProperties:124) 2022-08-23 05:35:07,426 INFO - [main:] ~ Loading atlas-application.properties from file:/data/hbase/conf/atlas-application.properties (ApplicationProperties:137) 2022-08-23 05:35:07,464 INFO - [main:] ~ Using graphdb backend 'janus' (ApplicationProperties:317) 2022-08-23 05:35:07,464 INFO - [main:] ~ Using storage backend 'hbase2' (ApplicationProperties:328) 2022-08-23 05:35:07,466 INFO - [main:] ~ Using index backend 'solr' (ApplicationProperties:339) 2022-08-23 05:35:07,467 INFO - [main:] ~ Atlas is running in MODE: PROD. (ApplicationProperties:343) 2022-08-23 05:35:07,467 INFO - [main:] ~ Setting solr.wait-searcher property 'true' (ApplicationProperties:350) 2022-08-23 05:35:07,468 INFO - [main:] ~ Setting index.search.map-name property 'false' (ApplicationProperties:354) 2022-08-23 05:35:07,468 INFO - [main:] ~ Setting atlas.graph.index.search.max-result-set-size = 150 (ApplicationProperties:364) 2022-08-23 05:35:07,468 INFO - [main:] ~ Setting atlas.graph.index.search.solr.wait-searcher = true (ApplicationProperties:365) 2022-08-23 05:35:07,468 INFO - [main:] ~ Property (set to default) atlas.graph.cache.db-cache = true (ApplicationProperties:377) 2022-08-23 05:35:07,468 INFO - [main:] ~ Property (set to default) atlas.graph.cache.db-cache-clean-wait = 20 (ApplicationProperties:377) 2022-08-23 05:35:07,468 INFO - [main:] ~ Property (set to default) atlas.graph.cache.db-cache-size = 0.5 (ApplicationProperties:377) 2022-08-23 05:35:07,468 INFO - [main:] ~ Property (set to default) atlas.graph.cache.tx-cache-size = 15000 (ApplicationProperties:377) 2022-08-23 05:35:07,469 INFO - [main:] ~ Property (set to default) atlas.graph.cache.tx-dirty-size = 120 (ApplicationProperties:377) 2022-08-23 05:35:07,476 ERROR - [main:] ~ ImportHBaseEntities failed (HBaseBridge:198) org.apache.atlas.AtlasException: Failed to load application properties at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:150) at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:103) at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:140) Caused by: org.apache.commons.configuration.ConversionException: 'atlas.graph.index.search.map-name' doesn't map to a List object: false, a java.lang.Boolean at org.apache.commons.configuration.AbstractConfiguration.getList(AbstractConfiguration.java:1144) at org.apache.commons.configuration.AbstractConfiguration.getList(AbstractConfiguration.java:1109) at org.apache.commons.configuration.AbstractConfiguration.interpolatedConfiguration(AbstractConfiguration.java:1274) at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:145) ... 2 more
... View more
Labels:
- Labels:
-
Apache Atlas
07-26-2022
01:49 AM
Hi Team, we are trying to integrate kafka and hbase into atlas 2.2.0 we are not using embedded kafka and hbase , seperately installed in same machine. kafka version : 2.13-2.5.0 atlas version : 2.2.0 /home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh --topic Maestrotopic3 >>>>> /home/ranger/apache-atlas-sources-2.2.0/hook-bin/import-kafka.sh >>>>> /home/ranger/apache-atlas-sources-2.2.0 Using Kafka configuration directory [/data/kafka_2.13-2.5.0/config] Log file for import is /var/log/atlas/import-kafka.log Picked up _JAVA_OPTIONS: -Xms1024m -Xmx1024m -XX:MaxPermSize=512m OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/data/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.clas s] SLF4J: Found binding in [jar:file:/data/kafka_2.13-2.5.0/libs/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory] Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/exception/NestableRuntimeException at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:756) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:473) at java.net.URLClassLoader.access$100(URLClassLoader.java:74) at java.net.URLClassLoader$1.run(URLClassLoader.java:369) at java.net.URLClassLoader$1.run(URLClassLoader.java:363) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:362) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:756) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:473) at java.net.URLClassLoader.access$100(URLClassLoader.java:74) at java.net.URLClassLoader$1.run(URLClassLoader.java:369) at java.net.URLClassLoader$1.run(URLClassLoader.java:363) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:362) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at org.apache.atlas.kafka.bridge.KafkaBridge.main(KafkaBridge.java:95) Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.exception.NestableRuntimeException at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 25 more Failed to import Kafka Data Model!!! FYI: kafka-console-producer.sh and kafka-console-consumer.sh are running fine. if we run the import-kafka.sh the without topic name , received similar error. Can you please help .
... View more
Labels:
- Labels:
-
Apache Atlas
