Support Questions

Find answers, ask questions, and share your expertise

Where to add timeout configuration for hive on spark

avatar

hive.spark.client.connect.timeout=90000;

I need to set this and I would like to set it in a configuration file and not in hql files. I do not see this property in hive-site.xml or spark xml.

Which files do I put this in?

For now, I have put it in:

Service Monitor Client Config Overrides

Is this the right place for it? Also, I don't have it in files anywhere so not sure if I am expected to add it in overrides.

1 ACCEPTED SOLUTION

avatar
@Sim kaur
  <property>
  <name>hive.spark.client.connect.timeout</name>
  <value>1000ms</value>
  <description>
  Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec,ns/nsec), which is msec if not specified. Timeout for remote Spark driver in connecting back to Hive client.
  </description>
  </property>
  <property>
  <name>hive.spark.client.server.connect.timeout</name>
  <value>90000ms</value>
  <description>
  Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is msec if not specified. Timeout for handshake between Hive client and remote Spark driver.  Checked by
both processes.
  </description>
  </property>

You can add the above properties in hive-site.xml. As the Spark will refer the hive-site.xml file, it will automatically gets updated in spark config. Hope this helps you.

View solution in original post

1 REPLY 1

avatar
@Sim kaur
  <property>
  <name>hive.spark.client.connect.timeout</name>
  <value>1000ms</value>
  <description>
  Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec,ns/nsec), which is msec if not specified. Timeout for remote Spark driver in connecting back to Hive client.
  </description>
  </property>
  <property>
  <name>hive.spark.client.server.connect.timeout</name>
  <value>90000ms</value>
  <description>
  Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is msec if not specified. Timeout for handshake between Hive client and remote Spark driver.  Checked by
both processes.
  </description>
  </property>

You can add the above properties in hive-site.xml. As the Spark will refer the hive-site.xml file, it will automatically gets updated in spark config. Hope this helps you.