Member since
06-07-2019
22
Posts
1
Kudos Received
0
Solutions
08-26-2019
07:11 PM
What are commands to bring to RW mode when VMs going into OS Read only mode. Can it be written in shell script
... View more
Labels:
- Labels:
-
Apache Hadoop
08-22-2019
03:45 PM
Can we use Ambari_helper concept to write the script to delete dead nodes from Ambari. Can please guide where Can I get details about Ambari_helper classes?
... View more
08-21-2019
03:12 PM
Can we have a shell script to delete the nodes in Ambari which were already deleted in Azure
... View more
Labels:
- Labels:
-
Apache Ambari
06-21-2019
04:10 PM
Thank you.As mentioned earlier netstat -tnlpa | grep `cat /var/run/hive/hive-server.pid` itself is not showing port 10001. Below is the error stack ERROR jdbc.HiveConnection: Error opening session org.apache.thrift.transport.TTransportException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:10001 [localhost/127.0.0.1] failed: Connection refused (Connection refused) at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:297) at org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) at org.apache.hive.service.cli.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:158) at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:150) at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:622) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:221) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:146) at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:211) at org.apache.hive.beeline.Commands.connect(Commands.java:1204) at org.apache.hive.beeline.Commands.connect(Commands.java:1100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:54) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:998) at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:717) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:779) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:493) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:476) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:243) at org.apache.hadoop.util.RunJar.main(RunJar.java:158) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:10001 [localhost/127.0.0.1] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.execchain.ServiceUnavailableRetryExec.execute(ServiceUnavailableRetryExec.java:84) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:117) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:251) ... 30 more Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134) ... 41 more Error: Could not establish connection to jdbc:hive2://localhost:10001/;transportMode=http: org.apache.http.conn.HttpHostConnectException: Connect to localhost:10001 [localhost/127.0.0.1] failed: Connection refused (Connection refused) (state=08S01,code=0)
... View more
06-19-2019
10:32 PM
1 Kudo
Hi Geoff, it is asking password to enter while I am using 'beeline> !connect jdbc:hive2://headnodehost:10001/;transportMode=http admin' Does this have different password
... View more
06-19-2019
05:19 PM
Thank you Jay for detailed reply. I need to property 'ActiveStandbyElector' from zookeeper stand point of view , like how it selects standby Resource manager to active one's. so you say, we can get this details by connecting to zookeeper and listing the contents in yarn? thank you once again
... View more
06-17-2019
11:37 PM
I am able to check this properties in the path /usr/hdp/current/hadoop-yarn-resourcemanager/etc/Hadoop/yarn-site <property> <name>yarn.resourcemanager.ha.automatic-failover.zk-base-path</name> <value>/yarn-leader-election</value> </property> <property> <name>yarn.resourcemanager.ha.enabled</name> <value>true</value> </property> am I looking correct path?
... View more
06-17-2019
11:12 PM
Hello, Where can we find (path) the property of ' ActiveStandbyElectorLock' or 'ActiveBreadcrumb' in zookeeper (path for zookeeper for ' ActiveStandbyElectorLock' or 'ActiveBreadcrumb')
... View more
- Tags:
- Hadoop Core
- Zookeeper
06-17-2019
07:25 PM
Hello, I am not able to identify where exactly this property is set for Yarn RM failover 'ActiveStandbyElector' for automatic failover Please provide the path and property where I can check this Thanks Nani
... View more
- Tags:
- Hadoop Core
- YARN
Labels:
- Labels:
-
Apache YARN
06-13-2019
06:25 PM
Thank you very much Jay. (1) What is the command to check if the process 'SPARK2_JOBHISTORYSERVER_PROCESS' is up and running (2) what is the port "spark2-defaults/spark.history.ui.port" ? basically is this only reason for this job failure
... View more
06-13-2019
06:25 PM
Thank you very much Jay. (1) What is the command to check if the process 'SPARK2_JOBHISTORYSERVER_PROCESS' is up and running (2) what is the port "spark2-defaults/spark.history.ui.port" ? basically is this only reason for this job failure
... View more
06-12-2019
10:41 PM
Hello, When Active Resource Manager and Stand by Resource Manager both are made down and when ZKFailoverController are up and running in all nodes. why ZKFailoverController is not able to bring resource manager to up. how can we bring resource manager to up ? is there any property to check Thanks Nani
... View more
Labels:
- Labels:
-
Apache YARN
06-10-2019
04:13 AM
Hello, Is there any way to create a custom script in bash to retrieve or get the custom alerts(Critical etc) from Ambari UI?
... View more
Labels:
- Labels:
-
Apache Ambari
06-09-2019
09:33 PM
Thank you Geoff. Please find the screen as attached. seems I am executing improperly.
... View more
06-09-2019
09:21 PM
For 1(a) this is the response " Unable to connect to remote host: Connection refused"
... View more
06-09-2019
06:39 PM
Also tried this step (Ambari UI --> Hive --> Summary (Tab) --> "HiveServer2 JDBC URL" --> Click on the at the right side of the URL to copy the URL and then try this URL with beeline once.) but it is showing no such file or directory found
... View more
06-09-2019
06:33 PM
Thank you. I completed step 1. (netstat -tnlpa | grep `cat /var/run/hive/hive-server.pid`) but could not find 10001. Trying to figure out the logs. Appreciate if you can share me the location of logs( hive configs and the Hive Logs). Also I was ale to connect through beeline 5 days back but not it is not letting me.
... View more
06-08-2019
05:04 PM
Hello, what is the purpose of Spark2_Jobhistoryserver_Process? . Under what circumstances we get alert for 'Spark2_Jobhistoryserver_Process' apart from stopping the service.
... View more
Labels:
- Labels:
-
Apache Spark
06-08-2019
05:03 PM
Hello, I am not able to connect to beeline. It is throwing error as "connection refused". I was able to see hive server2 service and hive metastore service is up and running through ambari UI. I am using the below command to connect beeline -u 'jdbc:hive2://headnodehost:10001/;transportMode=http' below is the error: Connecting to jdbc:hive2://headnodehost:10001/;transportMode=http
19/06/07 22:44:46 [main]: ERROR jdbc.HiveConnection: Error opening session
org.apache.thrift.transport.TTransportException: org.apache.http.conn.HttpHostConnectException: Connect to headnodehost:10001 [headnodehost/10.0.0.14] failed: Connection refused (Connection refused)
at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:297)
at org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73)
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
at org.apache.hive.service.cli.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:158)
at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:150)
at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:622)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:221)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:146)
at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:211)
at org.apache.hive.beeline.Commands.connect(Commands.java:1204)
at org.apache.hive.beeline.Commands.connect(Commands.java:1100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:54)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:998)
at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:717)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:779)
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:493)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:476)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:243)
at org.apache.hadoop.util.RunJar.main(RunJar.java:158)
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to headnodehost:10001 [headnodehost/10.0.0.14] failed: Connection refused (Connection refused)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.execchain.ServiceUnavailableRetryExec.execute(ServiceUnavailableRetryExec.java:84)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:117)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:251)
... 30 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
... 41 more
Error: Could not establish connection to jdbc:hive2://headnodehost:10001/;transportMode=http: org.apache.http.conn.HttpHostConnectException: Connect to headnodehost:10001 [headnodehost/10.0.0.14] failed: Connection refused (Connection refused) (state=08S01,code=0)
Beeline version 1.2.1000.2.6.5.3005-23 by Apache Hive
... View more
Labels:
- Labels:
-
Apache Hive