Hive meta store error when IS_RANGER flag is disabled in EMR restore.

[#1]

Hello Team,

When we trigger AWS EMR, cluster restore with IS_RANGER flag disabled then cluster get restore successfully but we try to execute hive table command we got errors as below - 

 Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: false Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:706) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:955) at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699) at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703) ... 9 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:987) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947) ... 15 more Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50) at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597) at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983) ... 16 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) ... 24 more Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused) at org.apache.thrift.transport.TSocket.open(TSocket.java:226) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:517) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50) at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597) at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947) at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699) at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.thrift.transport.TSocket.open(TSocket.java:221) ... 32 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:566) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) ... 29 more None 2023-10-20 05:09:42,787 - LANDING - ERROR - ERROR in execute_hive_script. - Error Message Process failed 1 Hive Session ID = 7320f657-08c1-4666-92c2-57a6364240ab Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: false Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:706) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:955) at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699) at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703) ... 9 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:987) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947) ... 15 more Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50) at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597) at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983) ... 16 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) ... 24 more Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused) at org.apache.thrift.transport.TSocket.open(TSocket.java:226) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:517) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50) at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597) at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947) at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699) at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.thrift.transport.TSocket.open(TSocket.java:221) ... 32 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:566) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) ... 29 more None 2023-10-20 05:09:42,787 - LANDING - ERROR - ERROR in {'self': <hub.spark_engine.edge.landing.landing_impl.landing_text_impl.LandingTextImpl object at 0x7fa9249bfd10>, 'e': Exception('Process failed 1 Hive Session ID = 7320f657-08c1-4666-92c2-57a6364240ab\n\nLogging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: false\nException in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient\n\tat org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:706)\n\tat org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616)\n\tat org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773)\n\tat org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.apache.hadoop.util.RunJar.run(RunJar.java:323)\n\tat org.apache.hadoop.util.RunJar.main(RunJar.java:236)\nCaused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient\n\tat org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:955)\n\tat org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699)\n\tat org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259)\n\tat org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703)\n\t... 9 more\nCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient\n\tat org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:987)\n\tat org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947)\n\t... 15 more\nCaused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient\n\tat org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121)\n\tat org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50)\n\tat org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597)\n\tat org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983)\n\t... 16 more\nCaused by: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\n\tat sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)\n\tat sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\n\tat java.lang.reflect.Constructor.newInstance(Constructor.java:423)\n\tat org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84)\n\t... 24 more\nCaused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)\n\tat org.apache.thrift.transport.TSocket.open(TSocket.java:226)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:517)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225)\n\tat org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94)\n\tat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\n\tat sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)\n\tat sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\n\tat java.lang.reflect.Constructor.newInstance(Constructor.java:423)\n\tat org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121)\n\tat org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50)\n\tat org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597)\n\tat org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983)\n\tat org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947)\n\tat org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699)\n\tat org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259)\n\tat org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703)\n\tat org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616)\n\tat org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773)\n\tat org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.apache.hadoop.util.RunJar.run(RunJar.java:323)\n\tat org.apache.hadoop.util.RunJar.main(RunJar.java:236)\nCaused by: java.net.ConnectException: Connection refused (Connection refused)\n\tat java.net.PlainSocketImpl.socketConnect(Native Method)\n\tat java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)\n\tat java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)\n\tat java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)\n\tat java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)\n\tat java.net.Socket.connect(Socket.java:607)\n\tat org.apache.thrift.transport.TSocket.open(TSocket.java:221)\n\t... 32 more\n)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:566)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225)\n\tat org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94)\n\t... 29 more\n None')} while running ERROR MESSAGE: Process failed 1 Hive Session ID = 7320f657-08c1-4666-92c2-57a6364240ab Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: false Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:706) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:955) at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699) at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703) ... 9 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:987) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947) ... 15 more Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50) at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597) at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983) ... 16 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) ... 24 more Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused) at org.apache.thrift.transport.TSocket.open(TSocket.java:226) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:517) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50) at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597) at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947) at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699) at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.thrift.transport.TSocket.open(TSocket.java:221) ... 32 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:566) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) ... 29 more None 2023-10-20 05:09:42,787 - LANDING - ERROR - ERROR in {'self': <hub.spark_engine.edge.executors.rdm_landing_orchestrator.RDMLandingOrchestrator object at 0x7fa93ce85f10>, 'landing_factory': <hub.spark_engine.edge.landing.landing_factory.LandingFactory object at 0x7fa924aa4a50>, 'landing': <hub.spark_engine.edge.landing.landing_impl.landing_text_impl.LandingTextImpl object at 0x7fa9249bfd10>, 'e': Exception('Process failed 1 Hive Session ID = 7320f657-08c1-4666-92c2-57a6364240ab\n\nLogging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: false\nException in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient\n\tat org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:706)\n\tat org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616)\n\tat org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773)\n\tat org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.apache.hadoop.util.RunJar.run(RunJar.java:323)\n\tat org.apache.hadoop.util.RunJar.main(RunJar.java:236)\nCaused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient\n\tat org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:955)\n\tat org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699)\n\tat org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259)\n\tat org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703)\n\t... 9 more\nCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient\n\tat org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:987)\n\tat org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947)\n\t... 15 more\nCaused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient\n\tat org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121)\n\tat org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50)\n\tat org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597)\n\tat org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983)\n\t... 16 more\nCaused by: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\n\tat sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)\n\tat sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\n\tat java.lang.reflect.Constructor.newInstance(Constructor.java:423)\n\tat org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84)\n\t... 24 more\nCaused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)\n\tat org.apache.thrift.transport.TSocket.open(TSocket.java:226)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:517)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225)\n\tat org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94)\n\tat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\n\tat sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)\n\tat sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)\n\tat java.lang.reflect.Constructor.newInstance(Constructor.java:423)\n\tat org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150)\n\tat org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121)\n\tat org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50)\n\tat org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617)\n\tat org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597)\n\tat org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983)\n\tat org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947)\n\tat org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699)\n\tat org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359)\n\tat org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259)\n\tat org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703)\n\tat org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616)\n\tat org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773)\n\tat org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.apache.hadoop.util.RunJar.run(RunJar.java:323)\n\tat org.apache.hadoop.util.RunJar.main(RunJar.java:236)\nCaused by: java.net.ConnectException: Connection refused (Connection refused)\n\tat java.net.PlainSocketImpl.socketConnect(Native Method)\n\tat java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)\n\tat java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)\n\tat java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)\n\tat java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)\n\tat java.net.Socket.connect(Socket.java:607)\n\tat org.apache.thrift.transport.TSocket.open(TSocket.java:221)\n\t... 32 more\n)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:566)\n\tat org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225)\n\tat org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94)\n\t... 29 more\n None')} while running ERROR MESSAGE: Process failed 1 Hive Session ID = 7320f657-08c1-4666-92c2-57a6364240ab Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: false Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:706) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:955) at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699) at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703) ... 9 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:987) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947) ... 15 more Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50) at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597) at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983) ... 16 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) ... 24 more Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused) at org.apache.thrift.transport.TSocket.open(TSocket.java:226) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:517) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50) at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597) at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947) at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699) at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.thrift.transport.TSocket.open(TSocket.java:221) ... 32 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:566) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) ... 29 more None 2023-10-20 05:09:42,787 - LANDING - ERROR - ERROR while running. ERROR MESSAGE: Process failed 1 Hive Session ID = 7320f657-08c1-4666-92c2-57a6364240ab Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j2.properties Async: false Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:706) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:955) at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699) at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703) ... 9 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:987) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947) ... 15 more Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50) at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597) at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983) ... 16 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) ... 24 more Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused) at org.apache.thrift.transport.TSocket.open(TSocket.java:226) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:517) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:97) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:150) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:121) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory.createMetaStoreClient(SessionHiveMetaStoreClientFactory.java:50) at org.apache.hadoop.hive.ql.metadata.HiveUtils.createMetaStoreClient(HiveUtils.java:490) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4617) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4597) at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:983) at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:947) at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1699) at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1310) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.setupSessionAcls(TezSessionState.java:583) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:359) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.beginOpen(TezSessionState.java:259) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:703) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:616) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:709) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:323) at org.apache.hadoop.util.RunJar.main(RunJar.java:236) Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.thrift.transport.TSocket.open(TSocket.java:221) ... 32 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:566) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:225) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:94) ... 29 more None 2023-10-20 05:09:42,788 - LANDING - ERROR - ZDHLandingExecutor script execution failed.

We use apache rangare version is 2.4 & EMR version is 5.36. Any way we can resolve this issue?

Regards,
Akshay

[#2]

@akshay_thite Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Ranger experts @Atahar @vamsi_redd  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3]

I don't see any error related to authorization here, is ranger authorization disabled for HMS?