Created 05-08-2018 01:19 AM
hive.spark.client.connect.timeout=90000;
I need to set this and I would like to set it in a configuration file and not in hql files. I do not see this property in hive-site.xml or spark xml.
Which files do I put this in?
For now, I have put it in:
Service Monitor Client Config Overrides
Is this the right place for it? Also, I don't have it in files anywhere so not sure if I am expected to add it in overrides.
Created 05-08-2018 03:06 AM
<property> <name>hive.spark.client.connect.timeout</name> <value>1000ms</value> <description> Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec,ns/nsec), which is msec if not specified. Timeout for remote Spark driver in connecting back to Hive client. </description> </property> <property> <name>hive.spark.client.server.connect.timeout</name> <value>90000ms</value> <description> Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is msec if not specified. Timeout for handshake between Hive client and remote Spark driver. Checked by both processes. </description> </property>
You can add the above properties in hive-site.xml. As the Spark will refer the hive-site.xml file, it will automatically gets updated in spark config. Hope this helps you.
Created 05-08-2018 03:06 AM
<property> <name>hive.spark.client.connect.timeout</name> <value>1000ms</value> <description> Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec,ns/nsec), which is msec if not specified. Timeout for remote Spark driver in connecting back to Hive client. </description> </property> <property> <name>hive.spark.client.server.connect.timeout</name> <value>90000ms</value> <description> Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is msec if not specified. Timeout for handshake between Hive client and remote Spark driver. Checked by both processes. </description> </property>
You can add the above properties in hive-site.xml. As the Spark will refer the hive-site.xml file, it will automatically gets updated in spark config. Hope this helps you.