I am following the document https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_installing_manually_book/content/startin... which talks about setting up of STS and starting the service.
In our Keerberized cluster, after successful service addition, when I try to start the service, I observed that it fails.
16/05/19 10:27:00 INFO AbstractService: Service:HiveServer2 is started. 16/05/19 10:27:00 INFO HiveThriftServer2: HiveThriftServer2 started 16/05/19 10:27:00 WARN SparkConf: The configuration key 'spark.yarn.applicationMaster.waitTries' has been deprecated as of Spark 1.3 and and may be removed in the future. Please use the new key 'spark.yarn.am.waitTime' instead. 16/05/19 10:27:00 INFO Server: jetty-8.y.z-SNAPSHOT 16/05/19 10:27:00 WARN AbstractLifeCycle: FAILED SelectChannelConnector@0.0.0.0:10001: java.net.BindException: Address already in use java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method)
When I observed the logs it says, port 10001 is already in use BindingException. In HiveServer2 I notice that for HTTP transport mode, bound port is 10001.
When I manually stopped HS2 on the same host and then started STS, then it works fine.
Here I've a few questions.
1) in the log it says that HS2 is started.. Is it trying to start HS2 ?
2) Port issue: Why still 10001 and not 10015?
For the STS, we have configured the port 10015 but when I try to connect using beeline : (After having a valid ticket)
beeline -u "jdbc:hive2://STSHOST:10015/default;httpPath=cliservice;transportMode=http;principal=hive/_HOST@Realm" it fails but when I try to connect on port 10001 it works (previously I used to connect to HS2 using the same command).
beeline -u "jdbc:hive2://STSHOST:10001/default;httpPath=cliservice;transportMode=http;principal=hive/_HOST@Realm"
And I can use STS to submit the sqls.
Could anyone please try to explain this behavior?
Can I have HS2 and STS running on the same node?
Can you be precise, which property to be changed? I do not want to change for HS2.
What I think is : Internally STS uses HS2 and not sure if it inherits port configurations of HS2.
Do you suggest to use command to start STS ?
./sbin/start-thriftserver.sh --master yarn-client --executor-memory 512m --hiveconf hive.server2.thrift.port=100015
I don't think we can run multiple instances of spark thrift from Ambari, Its better to run one instance from Ambari and another from command line.
./sbin/start-thriftserver.sh --master yarn-client --executor-memory 512m --hiveconf hive.server2.thrift.port=100015 &