- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Disable spark2 thrift server
- Labels:
-
Apache Ambari
-
Apache Hive
-
Apache Spark
Created on ‎12-02-2021 05:36 AM - edited ‎12-02-2021 05:45 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, since every time i run a job via spark-submit i have a delay due to the request attempt of spark to connect to hive metastore (and i don't need it since as i read is only for SparkSQL via JDBC) i'd like to disable this try of spark to connect to the metastore, how can i do it via ambari?
P.S. i have more spark client installed on different machines, 0 spark2 thrift server installed and 0 Livy for spark2 installed.
Created ‎01-07-2022 05:40 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Good Day!! Even though you submit spark-submit via JDBC the request goes through the HMS. HMS detects the type of client for interacting with it, for example, Hive or Spark, and acts accordingly. If you see any timeout requests in the logs you can adjust the below property.
hive.metastore.client.socket.timeout=<to a higher value>
Check and let us know If this helped.
