Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive meta data failed to import

avatar
Contributor

Hi, 

 

I am facing the below error when running the import-hive.sh to import hive metadata to Apache Atlas. 

 

2020-08-08T07:41:25,369 ERROR [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Import failed
com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) ~[jersey-client-1.19.jar:1.19]
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:382) ~[atlas-client-common-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353) ~[atlas-client-common-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:246) ~[atlas-client-common-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:285) ~[atlas-client-v2-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:276) ~[atlas-client-v2-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findEntity(HiveMetaStoreBridge.java:780) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findDatabase(HiveMetaStoreBridge.java:745) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:399) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:281) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:251) ~[hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:168) [hive-bridge-1.1.0.3.1.4.0-315.jar:1.1.0.3.1.4.0-315]
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_112]
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_112]
at java.net.SocketInputStream.read(SocketInputStream.java:170) ~[?:1.8.0_112]
at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_112]
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_112]
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[?:1.8.0_112]
at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_112]
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704) ~[?:1.8.0_112]
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) ~[?:1.8.0_112]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569) ~[?:1.8.0_112]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[?:1.8.0_112]
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[?:1.8.0_112]
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19]
... 16 more
Failed to import Hive Meta Data!!!

1 ACCEPTED SOLUTION

avatar
Contributor

This is solved now, guess something to do with memory. We increased the memory from 16GB to 32 GB on the EC2 instance, we were able to run all the services smoothly and were able to successfully run the import-hive.sh command without any error.

View solution in original post

1 REPLY 1

avatar
Contributor

This is solved now, guess something to do with memory. We increased the memory from 16GB to 32 GB on the EC2 instance, we were able to run all the services smoothly and were able to successfully run the import-hive.sh command without any error.