About jsensharma

jsensharma · ‎07-09-2018

@Anurag Mishra Something like this: # curl -s http://hdfcluster3.example.com:8088/ws/v1/cluster/scheduler | grep 'queueName' | python -c "import sys, json; print json.load(sys.stdin)['scheduler']['schedulerInfo']['queues']['queue'][0]['queueName']" # curl -s http://hdfcluster3.example.com:8088/ws/v1/cluster/scheduler | grep 'queueName' | python -c "import sys, json; print json.load(sys.stdin)['scheduler']['schedulerInfo']['queues']['queue'][1]['queueName']" # curl -s http://hdfcluster3.example.com:8088/ws/v1/cluster/scheduler | grep 'queueName' | python -c "import sys, json; print json.load(sys.stdin)['scheduler']['schedulerInfo']['queues']['queue'][2]['queueName']" Using a simple Shell script we can keep increasing the index in the same Curl command like ['queue'][3], ['queue'][4] .

jsensharma · ‎07-09-2018

@Anurag Mishra You can use Ambari CS view or RM UI. Usng command line if you want to get the details then you might want to try the following , however you will have to parse the output a bit # curl http://$ResourceManager_Address:8088/ws/v1/cluster/scheduler | grep --color 'queueName' .

jsensharma · ‎07-09-2018

@Michael Bronson Additionally please check if there is any issue in connecting to HiveSerevr2 using Beeline manually? This will give us some idea like how much time does the beeline connection takes. If it takes more than default hardcoded 30 secodns then please try to increase the timeout to a higher value like 60 in the same script: # grep 'timeout=30' /var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py # grep 'timeout=30' /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py .

jsensharma · ‎07-09-2018

@Michael Bronson If you see that there are many "beeline" processes are being created by ambari or ambari-qa user then it might be the Hive Alert checker scripts which might be leaving opened Beeline connections ... which might be causing the issue. Please see the script [1] On Ambari Server host /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py On Agent hosts /var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py . May be for some time you can try disabling the "HiveServer 2 Process" Alert from Ambari UI and then kill those beeling process manually using command line to see if it fixes the issue. Ideally there were some old issues reported like that but those were already supposed to be fixed ambari 2.5.0 onwards like: https://issues.apache.org/jira/browse/AMBARI-18286 . [1] https://github.com/apache/ambari/blob/release-2.6.1/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py#L275

jsensharma · ‎07-09-2018

@Michael Bronson HDP 2.6.4 to HDP 2.6.5 is a patch/maintenance HDP upgrade and should be normal upgrade. You can either choose Rolling / Express Upgrade option. Prerequisite will however remain same: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-upgrade/content/upgrading_hdp_stack.html

jsensharma · ‎07-04-2018

@Zeev Grinberg Please share the exact details about the exercise which is not working. So that we will know how that issue is related to HDP / Hortonworks products so that accordingly we can take a look.

jsensharma · ‎07-04-2018

@sudhir reddy How are you logging in to the Shell? Which command are you using to login in to the spark shell ? Which mode? For example when you run "spark-shell" then you will see lots of output ... we need that complete output. Example # spark-shell <br> . As much information you can provide about your issue ... there are possibilities that your issue will get resolved faster. Else there will be more interactions to know which HDP version are you using ? What is the logging happening? ...etc

jsensharma · ‎07-04-2018

@Kant T File View will allow to upload binary files, However you can only view/read the Text files via Ambari View. Binary files (Like PDF/ Image files) are not supposed to be displayed fine in File View. In order to view those files, you can however use the "Download" button present in the file view to download the file on your local machine (laptop) and then open it in the suitable s/w (like Image viewers/ PDF reader)

jsensharma · ‎07-04-2018

@sudhir reddy Can you please share the complete output along with the command that you ran?

jsensharma · ‎07-04-2018

@sudhir reddy Are you using Spark2? Have you set the Env variable as following before opening the shell? # export SPARK_MAJOR_VERSION=2 # spark-shell .

Member Since	‎03-14-2016 01:07 PM
Last Visited
Posts	4,721
Kudos received	1096

Cloudera Community

Re: set Variable in ambari rest API

Re: how to stop Hive Metastore and HiveServer2 by...

Re: how to verify by ambari api the active/standby...

Re: Curl throws error when running allow snapshot

Re: ambari server + REASON: Server not yet listeni...

Re: command to list all the yarn queues

Re: command to list all the yarn queues

Re: hive + many hive process was open on namenode ...

Re: hive + many hive process was open on namenode ...

Re: is it possible to upgrade HDP from 2.6.4 to 2....

Re: can't do the exercise on a mac machine

Re: How to use sqlContext? Getting an error!

Re: Can we open and see the pdf and jpeg files via...

Re: How to use sqlContext? Getting an error!

Re: How to use sqlContext? Getting an error!