Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Disable spark2 thrift server

avatar
Contributor

Hi, since every time i run a job via spark-submit i have a delay due to the request attempt of spark to connect to hive metastore (and i don't need it since as i read is only for SparkSQL via JDBC) i'd like to disable this try of spark to connect to the metastore, how can i do it via ambari? 
P.S. i have more spark client installed on different machines, 0 spark2 thrift server installed and 0 Livy for spark2 installed.

Thanks for your help!!
1 REPLY 1

avatar
Contributor

@Lo 

Good Day!! Even though you submit spark-submit via JDBC the request goes through the HMS. HMS detects the type of client for interacting with it, for example, Hive or Spark, and acts accordingly. If you see any timeout requests in the logs you can adjust the below property.

hive.metastore.client.socket.timeout=<to a higher value>

 Check and let us know If this helped.