Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP 3.1 Spark2 history server gets "Configuration parameter 'hive-env' was not found in configurations dictionary" on start

HDP 3.1 Spark2 history server gets "Configuration parameter 'hive-env' was not found in configurations dictionary" on start

New Contributor

Hi,

I am getting the following error on HDP 3.1 when starting up the Spark2 History server:

stderr:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/job_history_server.py", line 102, in <module>
JobHistoryServer().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/job_history_server.py", line 55, in start
spark_service('jobhistoryserver', upgrade_type=upgrade_type, action='start')
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/spark_service.py", line 105, in spark_service
create_catalog_cmd = format("{hive_schematool_bin}/schematool -dbType {hive_metastore_db_type} "
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/format.py", line 95, in format
return ConfigurationFormatter().format(format_string, args, **result)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/format.py", line 59, in format
result_protected = self.vformat(format_string, args, all_params)
File "/usr/lib64/python2.7/string.py", line 563, in vformat
result = self._vformat(format_string, args, kwargs, used_args, 2)
File "/usr/lib64/python2.7/string.py", line 596, in _vformat
result.append(self.format_field(obj, format_spec))
File "/usr/lib64/python2.7/string.py", line 613, in format_field
return format(value, format_spec)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/config_dictionary.py", line 73, in __getattr__
raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'hive-env' was not found in configurations dictionary!

I have installed HDP 3.1 via Ambari blueprints and the very same blueprint was working on HDP 2.6. My blueprint installs Zookeeper, HDFS, YARN and Spark2.


The strange thing is that if I start the spark history server from the script in the sbin directory of Spark, it starts up just fine.

Any help is appreciated as this is preventing our upgrade to HDP 3.1 from 2.x!

1 REPLY 1

Re: HDP 3.1 Spark2 history server gets "Configuration parameter 'hive-env' was not found in configurations dictionary" on start

New Contributor

Hi! Just wondering if someone could help me with this. Thanks!