atlas import-hive failed with exception

[#1]

Apache atlas failed with below exception while running import-hive.sh.

2017-05-08 14:35:42,975 DEBUG - [main:] ~ Using resource http://host1.hdp.com:21000/api/atlas/types for 0 times (AtlasClient:784) 2017-05-08 14:35:43,039 DEBUG - [main:] ~ API http://host1.hdp.com:21000/api/atlas/types returned status 400 (AtlasClient:1191) Exception in thread "main" org.apache.atlas.AtlasServiceException: Metadata service API CREATE_TYPE failed with status 400 (Bad Request) Response Body ({"error":"Unknown superType Referenceable in definition of type hive_storagedesc","stackTrace":"org.apache.atlas.AtlasException: Unknown superType Referenceable in definition of type hive_storagedesc\n\tat org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.validateSuperTypes(TypeSystem.java:406)\n\tat org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.step2(TypeSystem.java:433)\n\tat org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.defineTypes(TypeSystem.java:553)\n\tat org.apache.atlas.typesystem.types.TypeSystem.defineTypes(TypeSystem.java:244)\n\tat org.apache.atlas.typesystem.types.TypeSystem.defineTypes(TypeSystem.java:236)\n\tat org.apache.atlas.services.DefaultMetadataService.createType(DefaultMetadataService.java:166)\n\tat org.apache.atlas.web.resources.TypesResource.submit(TypesResource.java:84)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)\n\tat com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)\n\tat com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)\n\tat com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)\n\tat com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)\n\tat com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)\n\tat com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)\n\tat com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)\n\tat com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)\n\tat org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:67)\n\tat com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)\n\tat com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)\n\tat com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)\n\tat com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)\n\tat com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)\n\tat com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:499)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)\n\tat org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat java.lang.Thread.run(Thread.java:745)\n"}) at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:1209) at org.apache.atlas.AtlasClient.callAPIWithRetries(AtlasClient.java:785) at org.apache.atlas.AtlasClient.callAPI(AtlasClient.java:1214) at org.apache.atlas.AtlasClient.createType(AtlasClient.java:540) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:629) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:661) Failed to import Hive Data Model!!! [root@host1 bin]# ./import-hive.sh

[#2]

@Rajesh Reddy, the error says "Unknown superType Referenceable in definition of type hive_storagedesc". Referenceable is one of the system type in Atlas and is registered as a new type when Atlas starts for the first time. Can you copy the atlas application.log here to make sure if the type registration succeeded during atlas startup.

Can you also confirm if you can see the system type - "Referenceable" in http://<atlas_url>:21000/api/atlas/types call.

[#3]

applicationlog.tar.gz

@Sarath Subramanian applicationlog.tar.gz is the log file and below is the types api response.

{"results":["Process","fs_permissions","hdfs_path","Infrastructure","DataSet","fs_path","file_action"],"count":7,"requestId":"qtp996141683-11 - 901a6fbb-fc2f-4a74-8d74-ce9c58c39a11"}

[#4]

Just to make it clear, i have reinstalled atlas multiple times but getting stuck with the same exception.

[#5]

can you mention the version of Apache Atlas and HDP you are using?

[#6 · Accepted solution]

I upgraded my hdp from 2.4 to 2.5 and all went good. I was able to import data but the lineage is not visible. I will create a new question for this.