atlas import-hive failed with exception

brajeshreddy — Mon, 08 May 2017 16:27:33 GMT

Apache atlas failed with below exception while running import-hive.sh.

2017-05-08 14:35:42,975 DEBUG - [main:] ~ Using resource http://host1.hdp.com:21000/api/atlas/types for 0 times (AtlasClient:784) 2017-05-08 14:35:43,039 DEBUG - [main:] ~ API http://host1.hdp.com:21000/api/atlas/types returned status 400 (AtlasClient:1191) Exception in thread "main" org.apache.atlas.AtlasServiceException: Metadata service API CREATE_TYPE failed with status 400 (Bad Request) Response Body ({"error":"Unknown superType Referenceable in definition of type hive_storagedesc","stackTrace":"org.apache.atlas.AtlasException: Unknown superType Referenceable in definition of type hive_storagedesc\n\tat org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.validateSuperTypes(TypeSystem.java:406)\n\tat org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.step2(TypeSystem.java:433)\n\tat org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.defineTypes(TypeSystem.java:553)\n\tat org.apache.atlas.typesystem.types.TypeSystem.defineTypes(TypeSystem.java:244)\n\tat org.apache.atlas.typesystem.types.TypeSystem.defineTypes(TypeSystem.java:236)\n\tat org.apache.atlas.services.DefaultMetadataService.createType(DefaultMetadataService.java:166)\n\tat org.apache.atlas.web.resources.TypesResource.submit(TypesResource.java:84)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:606)\n\tat com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)\n\tat com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)\n\tat com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)\n\tat com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)\n\tat com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)\n\tat com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)\n\tat com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)\n\tat com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)\n\tat com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)\n\tat com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)\n\tat org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:67)\n\tat com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)\n\tat com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)\n\tat com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)\n\tat com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)\n\tat com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)\n\tat com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:499)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)\n\tat org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat java.lang.Thread.run(Thread.java:745)\n"}) at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:1209) at org.apache.atlas.AtlasClient.callAPIWithRetries(AtlasClient.java:785) at org.apache.atlas.AtlasClient.callAPI(AtlasClient.java:1214) at org.apache.atlas.AtlasClient.createType(AtlasClient.java:540) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:629) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:661) Failed to import Hive Data Model!!! [root@host1 bin]# ./import-hive.sh

Re: atlas import-hive failed with exception

ssubramanian — Mon, 08 May 2017 22:56:26 GMT

@Rajesh Reddy, the error says "Unknown superType Referenceable in definition of type hive_storagedesc". Referenceable is one of the system type in Atlas and is registered as a new type when Atlas starts for the first time. Can you copy the atlas application.log here to make sure if the type registration succeeded during atlas startup.

Can you also confirm if you can see the system type - "Referenceable" in http://<atlas_url>:21000/api/atlas/types call.

Re: atlas import-hive failed with exception

brajeshreddy — Tue, 09 May 2017 12:22:46 GMT

applicationlog.tar.gz

@Sarath Subramanian applicationlog.tar.gz is the log file and below is the types api response.

{"results":["Process","fs_permissions","hdfs_path","Infrastructure","DataSet","fs_path","file_action"],"count":7,"requestId":"qtp996141683-11 - 901a6fbb-fc2f-4a74-8d74-ce9c58c39a11"}

Re: atlas import-hive failed with exception

brajeshreddy — Tue, 09 May 2017 12:45:51 GMT

Just to make it clear, i have reinstalled atlas multiple times but getting stuck with the same exception.

Re: atlas import-hive failed with exception

ssubramanian — Tue, 09 May 2017 23:18:58 GMT

can you mention the version of Apache Atlas and HDP you are using?

Re: atlas import-hive failed with exception

brajeshreddy — Thu, 11 May 2017 09:40:26 GMT

I upgraded my hdp from 2.4 to 2.5 and all went good. I was able to import data but the lineage is not visible. I will create a new question for this.

question Re: atlas import-hive failed with exception in Archives of Support Questions (Read Only)

atlas import-hive failed with exception

Re: atlas import-hive failed with exception

Re: atlas import-hive failed with exception

Re: atlas import-hive failed with exception

Re: atlas import-hive failed with exception

Re: atlas import-hive failed with exception