Support Questions
Find answers, ask questions, and share your expertise

Failed to import Hive Meta Data

New Contributor

Hi, i'm looking for some help about atlas

when i try to run import-hive.sh to import data from hive to atlas, it gives NoClassDefFoundError

Using Hive configuration directory [/usr/hdp/2.6.4.0-91/hive/conf]
/usr/hdp/2.6.4.0-91/hadoop/conf:/usr/hdp/2.6.4.0-91/hadoop/lib/*:/usr/hdp/2.6.4.0-91/hadoop/.//*:/usr/hdp/2.6.4.0-91/hadoop-hdfs/./:/usr/hdp/2.6.4.0-91/hadoop-hdfs/lib/*:/usr/hdp/2.6.4.0-91/hadoop-hdfs/.//*:/usr/hdp/2.6.4.0-91/hadoop-yarn/lib/*:/usr/hdp/2.6.4.0-91/hadoop-yarn/.//*:/usr/hdp/2.6.4.0-91/hadoop-mapreduce/lib/*:/usr/hdp/2.6.4.0-91/hadoop-mapreduce/.//*::mysql-connector-java.jar:/usr/hdp/2.6.4.0-91/tez/*:/usr/hdp/2.6.4.0-91/tez/lib/*:/usr/hdp/2.6.4.0-91/tez/conf
Log file for import is /data0/apache-atlas-1.1.0/logs/import-hive.log
log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout.
Enter username for atlas :- admin
Enter password for atlas :- 
Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/jaxrs/json/JacksonJaxbJsonProvider
        at org.apache.atlas.AtlasBaseClient.getClient(AtlasBaseClient.java:253)
        at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:425)
        at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:420)
        at org.apache.atlas.AtlasBaseClient.<init>(AtlasBaseClient.java:115)
        at org.apache.atlas.AtlasClientV2.<init>(AtlasClientV2.java:77)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:131)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 6 more
Failed to import Hive Meta Data!!!

i try to put jackson-jaxrs-json-provider.jar under atlas/hook/hive/atlas-hive-plugin-impl

then it gives me this

Using Hive configuration directory [/usr/hdp/2.6.4.0-91/hive/conf]
/usr/hdp/2.6.4.0-91/hadoop/conf:/usr/hdp/2.6.4.0-91/hadoop/lib/*:/usr/hdp/2.6.4.0-91/hadoop/.//*:/usr/hdp/2.6.4.0-91/hadoop-hdfs/./:/usr/hdp/2.6.4.0-91/hadoop-hdfs/lib/*:/usr/hdp/2.6.4.0-91/hadoop-hdfs/.//*:/usr/hdp/2.6.4.0-91/hadoop-yarn/lib/*:/usr/hdp/2.6.4.0-91/hadoop-yarn/.//*:/usr/hdp/2.6.4.0-91/hadoop-mapreduce/lib/*:/usr/hdp/2.6.4.0-91/hadoop-mapreduce/.//*::mysql-connector-java.jar:/usr/hdp/2.6.4.0-91/tez/*:/usr/hdp/2.6.4.0-91/tez/lib/*:/usr/hdp/2.6.4.0-91/tez/conf
Log file for import is /data0/apache-atlas-1.1.0/logs/import-hive.log
log4j:WARN No such property [maxFileSize] in org.apache.log4j.PatternLayout.
log4j:WARN No such property [maxBackupIndex] in org.apache.log4j.PatternLayout.
Enter username for atlas :- admin
Enter password for atlas :- 
Exception in thread "main" java.lang.AbstractMethodError: com.fasterxml.jackson.jaxrs.base.ProviderBase._configForWriting(Lcom/fasterxml/jackson/databind/ObjectMapper;[Ljava/lang/annotation/Annotation;)Lcom/fasterxml/jackson/jaxrs/cfg/EndpointConfigBase;
        at com.fasterxml.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:490)
        at com.sun.jersey.api.client.RequestWriter.writeRequestEntity(RequestWriter.java:300)
        at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:204)
        at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
        at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:81)
        at com.sun.jersey.api.client.Client.handle(Client.java:648)
        at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
        at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
        at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
        at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:356)
        at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:327)
        at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:212)
        at org.apache.atlas.AtlasClientV2.createEntity(AtlasClientV2.java:285)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerInstance(HiveMetaStoreBridge.java:446)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:398)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:277)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:247)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:168)
Failed to import Hive Meta Data!!!

here is the import-hive.log

2019-03-22 02:00:00,665 WARN  - [main:] ~ Unable to load native-hadoop library for your platform... using builtin-java classes where applicable (NativeCodeLoader:62)
2019-03-22 02:00:03,654 INFO  - [main:] ~ Importing Hive metadata (HiveMetaStoreBridge:245)
2019-03-22 02:00:03,670 INFO  - [main:] ~ Found 2 databases (HiveMetaStoreBridge:274)
2019-03-22 02:00:03,713 INFO  - [main:] ~ method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=404 (AtlasBaseClient:361)
2019-03-22 02:00:31,924 INFO  - [main:] ~ Looking for atlas-application.properties in classpath (ApplicationProperties:85)
2019-03-22 02:00:31,928 INFO  - [main:] ~ Loading atlas-application.properties from file:/etc/hive/2.6.4.0-91/0/atlas-application.properties (ApplicationProperties:98)
2019-03-22 02:00:31,965 INFO  - [main:] ~ Property (set to default) atlas.graph.cache.db-cache = true (ApplicationProperties:242)
2019-03-22 02:00:31,965 INFO  - [main:] ~ Property (set to default) atlas.graph.cache.db-cache-clean-wait = 20 (ApplicationProperties:242)
2019-03-22 02:00:31,967 INFO  - [main:] ~ Property (set to default) atlas.graph.cache.db-cache-size = 0.5 (ApplicationProperties:242)
2019-03-22 02:00:31,968 INFO  - [main:] ~ Property (set to default) atlas.graph.cache.tx-cache-size = 15000 (ApplicationProperties:242)
2019-03-22 02:00:31,968 INFO  - [main:] ~ Property (set to default) atlas.graph.cache.tx-dirty-size = 120 (ApplicationProperties:242)
2019-03-22 02:00:35,595 INFO  - [main:] ~ Client has only one service URL, will use that for all actions: http://localhost:21000 (AtlasBaseClient:295)

I'd be very grateful if someone can help me out of this