Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Atlas import from hive FAILED with import-hive.sh

avatar
Explorer

I use Apache Atlas from HDP 2.6.3

While i run import-hive.sh, this script have imported any databases and tables(not all), but interrupted by next exception:

2018-03-29 11:14:27,257 INFO - [main:] ~ Importing objects from DB.table1 (HiveMetaStoreBridge:433) 2018-03-29 11:15:27,356 ERROR - [main:] ~ Import failed for hive_table table1 (HiveMetaStoreBridge:326) org.apache.atlas.hook.AtlasHookException: HiveMetaStoreBridge.getStorageDescQFName() failed. at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaStoreBridge.java:515) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importTable(HiveMetaStoreBridge.java:289) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importTables(HiveMetaStoreBridge.java:272) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:143) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:134) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:647) Caused by: com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) at com.sun.jersey.api.client.Client.handle(Client.java:648) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:334) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:311) at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:199) at org.apache.atlas.AtlasClient.callAPIWithBodyAndParams(AtlasClient.java:952) at org.apache.atlas.AtlasClient.updateEntity(AtlasClient.java:505) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.updateInstance(HiveMetaStoreBridge.java:526) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaStoreBridge.java:511) ... 5 more Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:240) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147) ... 16 more

2018-03-29 10:33:37,303 ERROR - [main:] ~ Able to import 4 tables out of 31 tables from DB. Please check logs for import errors (HiveMetaStoreBridge:279)

I suppose that need increase some timeout , because this interrupted after 60 000 ms... What is timeout controls this?

1 ACCEPTED SOLUTION

avatar
Explorer

I have resolved this issue! This issue happens because, HDP 2.6.3 consist Atlas old version, and it does't have appropriate timeout parameters. This bug was fix in https://issues.apache.org/jira/browse/ATLAS-690

I bulid Atlas with master branch and run import with new Jars. And added this parameter : atlas.client.readTimeoutMSecs=600000

And all works propertly, without timeout exeptions.

View solution in original post

1 REPLY 1

avatar
Explorer

I have resolved this issue! This issue happens because, HDP 2.6.3 consist Atlas old version, and it does't have appropriate timeout parameters. This bug was fix in https://issues.apache.org/jira/browse/ATLAS-690

I bulid Atlas with master branch and run import with new Jars. And added this parameter : atlas.client.readTimeoutMSecs=600000

And all works propertly, without timeout exeptions.