Livy issue while HDP 2.6.2.0 upgrade

[#1]

I'm upgrading from HDP 2.4.2.0 to 2.6.2.0. All tasks from Ambari wizard were performed OK but Spark fails.

I got this error message while restarting Spark Thrift Server

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/spark_thrift_server.py", line 87, in <module>
    SparkThriftServer().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 830, in restart
    self.stop(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/spark_thrift_server.py", line 57, in stop
    import params
  File "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/params.py", line 262, in <module>
    livy_principal = livy_kerberos_principal.replace('_HOST', config['hostname'].lower())
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py", line 73, in __getattr__
    raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'livy.server.launch.kerberos.principal' was not found in configurations dictionary!

I just added livy.server.launch.kerberos.principal in /etc/livy/conf/livy-defaults.conf but doesn't work.

What should I do? Thanks in advance!

[#2]

@Facundo Bianco

I remember a very similar issue i faced some times back.

Can you please check your "clusterstate" and "stack" table in the ambari DB?

# psql -U ambari ambari
Password for user ambari: bigdata
psql (9.2.18)
Type "help" for help.

ambari=> select * from clusterstate where current_stack_id=2;
 cluster_id | current_cluster_state | current_stack_id 
------------+-----------------------+------------------
          2 |                       |                2
(1 row)

ambari=> select * from stack where stack_id=2;
 stack_id | stack_name | stack_version 
----------+------------+---------------
        2 | HDP        | 2.6
(1 row)

.

The Stack ID =2 should be pointing to the HDP 2.6 stack and also the clusterstate table record should have the same current_stack_id =2.

If you find any difference then you should collect a DB dump of ambari and then update the "clusterstate" table to point to the correct stack_id (which is 2 for HDP 2.6 clusters).

.

This is just a guess at this point, else we will have to look at the livy conf and the upgrade logs.