Oozie Service Check fails after upgrading to Ambari-2.7.3: raise Fail(format(NO_DOCS_FOLDER_MESSAGE)) NameError: global name 'Fail' is not defined

[#1]

HDP-2.6.5, RHEL 7, Kerberized, Ambari-2.6 upgraded to Ambari-2.7.3

Oozie Service Check fails:

STDERR

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py", line 139, in <module>
    OozieServiceCheck().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py", line 52, in service_check
    OozieServiceCheckDefault.oozie_smoke_shell_file(smoke_test_file_name, prepare_hdfs_file_name)
  File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py", line 70, in oozie_smoke_shell_file
    raise Fail(format(NO_DOCS_FOLDER_MESSAGE)) 
NameError: global name 'Fail' is not defined

STDOUT

2019-02-15 19:02:10,245 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=2.6.5.74-2 -> 2.6.5.74-2
2019-02-15 19:02:10,250 - Using hadoop conf dir: /usr/hdp/2.6.5.74-2/hadoop/conf
2019-02-15 19:02:10,275 - File['/var/lib/ambari-agent/cred/lib/CredentialUtil.jar'] {'content': DownloadSource('http://ambariServer.com:8080/resources/CredentialUtil.jar'), 'mode': 0755}
2019-02-15 19:02:10,277 - Not downloading the file from http://ambariServer:8080/resources/CredentialUtil.jar, because /var/lib/ambari-agent/tmp/CredentialUtil.jar already exists
2019-02-15 19:02:11,274 - File['/var/lib/ambari-agent/tmp/oozieSmoke2.sh'] {'content': StaticFile('oozieSmoke2.sh'), 'mode': 0755}
2019-02-15 19:02:11,277 - File['/var/lib/ambari-agent/tmp/prepareOozieHdfsDirectories.sh'] {'content': StaticFile('prepareOozieHdfsDirectories.sh'), 'mode': 0755}

Command failed after 1 tries

I have noticed that hosts with Oozie Server and Clients do not have the following directories:

1. /usr/hdp/current/oozie-client/doc/
2. /usr/hdp/current/oozie-server/doc/

So, I have tried removing tsflags=nodocs from /etc/yum.conf and attempted reinstalling the oozie client, however, it still does not create the above folders.

[#2]

@Kuldeep Kulkarni, @amarnath reddy pappu

Any pointers? TIA.

[#3]

@Dinesh Chitlangia

On the host where you are seeing the issue, try removing the rpm's of oozie using yum remove oozie_2_6_* and then try reinstalling it from the command line using the command yum install oozie_2_6_*

[#4]

@Mahesh Balakrishnan Since there can be only one accepted answer 😞 , I am sharing 25 bounty points with you. Thanks for the guidance.

[#5 · Accepted solution]

Please modify below script on ambari-server and replace cached copy of the same script with the modified script on each ambari-agent ( where you have oozie clients installed )

Script to be modified on Ambari Server:

/var/lib/ambari-server/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py

What to modify?

Add below line:

from resource_management import *

Where is cached copy on ambari-agents/oozie-clients?

/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py

After this, you may face below error:

resource_management.core.exceptions.Fail: Cannot find /usr/hdp/current/oozie-client/doc. Possible reason is that /etc/yum.conf contains tsflags=nodocs which prevents this folder from being installed along with oozie-client package. If this is the case, please fix /etc/yum.conf and re-install the package.

To fix this,

Edit /etc/yum.conf and remove/comment out

tsflag=nodocs 

Run below commands to reinstall oozie packages.

yum reinstall -y oozie_2_6_*

Hope this helps! 🙂

[#6]

@Kuldeep Kulkarni

Thanks that got me past the initial problem and then I ran into following error:

resource_management.core.exceptions.Fail: Cannot find /usr/hdp/current/oozie-client/doc. Possible reason is that /etc/yum.conf contains tsflags=nodocs which prevents this folder from being installed along with oozie-client package. If this is the case, please fix /etc/yum.conf and re-install the package.

This was resolved by @Mahesh Balakrishnan's input of remove and install the packages.