Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Atlas import from hive FAILED with import-hive.sh

avatar

I use Apache Atlas from HDP 2.6.3

While i run import-hive.sh, this script have imported any databases and tables(not all), but interrupted by next exception:

2018-03-29 11:14:27,257 INFO - [main:] ~ Importing objects from DB.table1 (HiveMetaStoreBridge:433) 2018-03-29 11:15:27,356 ERROR - [main:] ~ Import failed for hive_table table1 (HiveMetaStoreBridge:326) org.apache.atlas.hook.AtlasHookException: HiveMetaStoreBridge.getStorageDescQFName() failed. at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaStoreBridge.java:515) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importTable(HiveMetaStoreBridge.java:289) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importTables(HiveMetaStoreBridge.java:272) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:143) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:134) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:647) Caused by: com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149) at com.sun.jersey.api.client.Client.handle(Client.java:648) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:334) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:311) at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:199) at org.apache.atlas.AtlasClient.callAPIWithBodyAndParams(AtlasClient.java:952) at org.apache.atlas.AtlasClient.updateEntity(AtlasClient.java:505) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.updateInstance(HiveMetaStoreBridge.java:526) at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaStoreBridge.java:511) ... 5 more Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:240) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147) ... 16 more

2018-03-29 10:33:37,303 ERROR - [main:] ~ Able to import 4 tables out of 31 tables from DB. Please check logs for import errors (HiveMetaStoreBridge:279)

I suppose that need increase some timeout , because this interrupted after 60 000 ms... What is timeout controls this?

1 ACCEPTED SOLUTION

avatar

I have resolved this issue! This issue happens because, HDP 2.6.3 consist Atlas old version, and it does't have appropriate timeout parameters. This bug was fix in https://issues.apache.org/jira/browse/ATLAS-690

I bulid Atlas with master branch and run import with new Jars. And added this parameter : atlas.client.readTimeoutMSecs=600000

And all works propertly, without timeout exeptions.

View solution in original post

1 REPLY 1

avatar

I have resolved this issue! This issue happens because, HDP 2.6.3 consist Atlas old version, and it does't have appropriate timeout parameters. This bug was fix in https://issues.apache.org/jira/browse/ATLAS-690

I bulid Atlas with master branch and run import with new Jars. And added this parameter : atlas.client.readTimeoutMSecs=600000

And all works propertly, without timeout exeptions.