Member since
06-15-2017
5
Posts
0
Kudos Received
0
Solutions
11-04-2017
09:37 PM
1 Kudo
@Balsher Singh The Easiest way will be to find the port using Ambari UI Login to Ambari UI --> Spark2 --> Configs (Tab) --> Advanced (Sub Tab) --> Advanced spark2-hive-site-override
(OR)
Login to Ambari UI --> Spark --> Configs (Tab) --> Advanced (Sub Tab) --> Advanced spark-hive-site-override
. The default Spark Thrift server port is 10015 (for Spark2 10016). To specify a different port, you can navigate to the hive.server2.thrift.port setting in the "Advanced spark-hive-site-override" category of the Spark configuration section and update the setting with your preferred port number. https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_spark-component-guide/content/config-sts-port.html . You can also use Ambari API to find the port using curl call as following: # curl -u admin:admin -i -H 'X-Requested-By: ambari' -X GET "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=spark2-hive-site-override"
(OR)
# curl -u admin:admin -i -H 'X-Requested-By: ambari' -X GET "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=spark-hive-site-override" Above command will list the various tags. You need to use the latest Tag ID (like "tag=version1509830820763") and then run the command with that tag ID as following: # curl -u admin:admin -H 'X-Requested-By: ambari' -X GET "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=spark2-hive-site-override&tag=version1509830820763"
{
"href" : "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=spark2-hive-site-override&tag=version1509830820763",
"items" : [
{
"href" : "http://localhost:8080/api/v1/clusters/Sandbox/configurations?type=spark2-hive-site-override&tag=version1509830820763",
"tag" : "version1509830820763",
"type" : "spark2-hive-site-override",
"version" : 2,
"Config" : {
"cluster_name" : "Sandbox",
"stack_id" : "HDP-2.6"
},
"properties" : {
"hive.metastore.client.connect.retry.delay" : "5",
"hive.metastore.client.socket.timeout" : "1800",
"hive.server2.enable.doAs" : "false",
"hive.server2.thrift.port" : "10016",
"hive.server2.transport.mode" : "binary"
}
}
]
} . NOTE: Please make sure that you put the whole URL inside Quotation mark as it contains & symbol in it. Another option will be to use the config.sh , you can find the port as following by running the below command from Ambari Server Host: For Spark2 # /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin get localhost Sandbox spark2-hive-site-override
OUTPUT --------
USERID=admin
PASSWORD=admin
########## Performing 'GET' on (Site:spark2-hive-site-override, Tag:version1509830820763)
"properties" : {
"hive.metastore.client.connect.retry.delay" : "5",
"hive.metastore.client.socket.timeout" : "1800",
"hive.server2.enable.doAs" : "false",
"hive.server2.thrift.port" : "10017",
"hive.server2.transport.mode" : "binary"
} . For Old Spark. # /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin get localhost Sandbox spark-hive-site-override
OUTPUT
--------
USERID=admin
PASSWORD=admin
########## Performing 'GET' on (Site:spark-hive-site-override, Tag:INITIAL)
"properties" : {
"hive.metastore.client.connect.retry.delay" : "5",
"hive.metastore.client.socket.timeout" : "1800",
"hive.server2.enable.doAs" : "false",
"hive.server2.thrift.port" : "10015",
"hive.server2.transport.mode" : "binary"
} . NOTE: In the above commands please replace "Sandbox" word with yoru HDP ClusterName. "localhost" with your ambari server hostname. .
... View more