Skip to main content
ashish_inamdar
ashish_inamdar
Gold ⭐️
August 8, 2020
Solved

Hive meta data failed to import

Hi, 

 

I am facing the below error when running the import-hive.sh to import hive metadata to Apache Atlas. 

 

2020-08-08T07:41:25,369 ERROR [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Import failed
com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) ~[jersey-client-1.19.jar:1.19]
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:382) ~[atlas-client-common-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353) ~[atlas-client-common-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:246) ~[atlas-client-common-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:285) ~[atlas-client-v2-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:276) ~[atlas-client-v2-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findEntity(HiveMetaStoreBridge.java:780) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findDatabase(HiveMetaStoreBridge.java:745) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:399) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:281) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:251) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:168) [hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_112]
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_112]
at java.net.SocketInputStream.read(SocketInputStream.java:170) ~[?:1.8.0_112]
at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_112]
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_112]
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[?:1.8.0_112]
at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_112]
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704) ~[?:1.8.0_112]
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) ~[?:1.8.0_112]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569) ~[?:1.8.0_112]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[?:1.8.0_112]
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[?:1.8.0_112]
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19]
... 16 more
Failed to import Hive Meta Data!!!

ashish_inamdar
ashish_inamdarAnswer
Gold ⭐️
August 11, 2020

This is solved now, guess something to do with memory. We increased the memory from 16GB to 32 GB on the EC2 instance, we were able to run all the services smoothly and were able to successfully run the import-hive.sh command without any error.

View in thread

1 reply

ashish_inamdar
ashish_inamdarAuthorAnswer
Gold ⭐️
August 11, 2020

This is solved now, guess something to do with memory. We increased the memory from 16GB to 32 GB on the EC2 instance, we were able to run all the services smoothly and were able to successfully run the import-hive.sh command without any error.

Translate
Terms & ConditionsAccessibility statement