Created on 08-31-2018 06:12 AM - edited 09-16-2022 06:39 AM
Hi,
We have recently upgraded from 5.11.2 to 5.14.2. The upgrade was untill we identified that many databases were not visible/loaded in navigator. The logs have below. Any related pointers are welcome.
2018-08-31 10:12:17,344 ERROR com.cloudera.nav.hive.extractor.AbstractHiveExtractor [CDHExecutor-0-CDHUrlClassLoader@7fbf5fdd]: Failed to extract database dummy_database with error: java.net.SocketException: Broken pipe (Write failed)
org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe (Write failed)
at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147)
at org.apache.thrift.transport.TTransport.write(TTransport.java:107)
at org.apache.thrift.transport.TSaslTransport.writeLength(TSaslTransport.java:391)
at org.apache.thrift.transport.TSaslTransport.flush(TSaslTransport.java:499)
at org.apache.thrift.transport.TSaslClientTransport.flush(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.TFilterTransport.flush(TFilterTransport.java:77)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_get_database(ThriftHiveMetastore.java:664)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:656)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:1213)
at com.cloudera.nav.hive.extractor.AbstractHiveExtractor.extractDatabase(AbstractHiveExtractor.java:147)
at com.cloudera.nav.hive.extractor.AbstractHiveExtractor.extractDatabases(AbstractHiveExtractor.java:133)
at com.cloudera.nav.hive.extractor.HiveExtractor.run(HiveExtractor.java:63)
at com.cloudera.nav.hive.extractor.AbstractHiveExtractor.run(AbstractHiveExtractor.java:118)
at com.cloudera.nav.hive.extractor.HiveExtractorShim.run(HiveExtractorShim.java:35)
at com.cloudera.cmf.cdhclient.CdhExecutor$RunnableWrapper.call(CdhExecutor.java:221)
at com.cloudera.cmf.cdhclient.CdhExecutor$RunnableWrapper.call(CdhExecutor.java:211)
at com.cloudera.cmf.cdhclient.CdhExecutor$CallableWrapper.doWork(CdhExecutor.java:236)
at com.cloudera.cmf.cdhclient.CdhExecutor$SecurityWrapper$1.run(CdhExecutor.java:189)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
at com.cloudera.cmf.cdh5client.security.UserGroupInformationImpl.doAs(UserGroupInformationImpl.java:44)
at com.cloudera.cmf.cdhclient.CdhExecutor$SecurityWrapper.doWork(CdhExecutor.java:186)
at com.cloudera.cmf.cdhclient.CdhExecutor$1.call(CdhExecutor.java:125)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
Thanks
Created 10-31-2018 02:16 AM
This is because navigator upgrade generally takes lot of time based on the number of objects and relations you have. Increasing the navigator heap size can help. Calculation of the required heap is available in cloudera site.
Created 10-31-2018 02:16 AM
This is because navigator upgrade generally takes lot of time based on the number of objects and relations you have. Increasing the navigator heap size can help. Calculation of the required heap is available in cloudera site.