Restart required for Atlas and Oozie after install of NiFi

[#1]

After installing NiFi, Oozie Server and Atlas Metadata Server components require a restart. Restart is failing. oozie-error.log has the following:

ERROR ShareLibService:517 ... E0104: Could not fully initiLize service ShareLibService, Not able to cache sharelib. An Admin needs to install the sharelib with oozie-setup.sh and issue the 'oozie admin' CLI command to update the sharelib

[#2]

However, it appears that a sharelib already exists as revealed by

hdfs dfs -ls /user/oozie/share/lib
Found 1 items
drwxr-xr-x   - oozie hdfs          0 2017-07-28 14:06 /user/oozie/share/lib/lib_20170728140342

[#3]

@Dave Welden

It seems your oozie is picking old libraries can you do the following

As root

# su - oozie

Check the sharedlibs below is output from my cluster

$ oozie admin -shareliblist 
[Available ShareLib] 
hive 
distcp 
mapreduce-streaming 
spark oozie 
hcatalog 
hive2 
sqoop 
pig 
spark_orig 

Run the update

$ oozie admin -sharelibupdate 
[ShareLib update status] 
sharelibDirOld = hdfs://test.com:8020/user/oozie/share/lib/lib_20170731003540 
host = http://test.com:11000/oozie 
sharelibDirNew = hdfs://test.com:8020/user/oozie/share/lib/lib_20170731003540 
status = Successful

Then retry start the oozie

[#4]

Hi @Geoffrey Shelton Okot

I tried the above as follows:

sandbox login: root
root@sandbox.hortonworks.com's password:
[root@sandbox ~]# su - oozie
[oozie@sandbox ~]$ oozie admin -shareliblist
[Available ShareLib]


[oozie@sandbox ~]$ oozie admin -sharelibupdate
[ShareLib update status]
        sharelibDirOld = null
        host = http://sandbox.hortonworks.com:11000/oozie
        sharelibDirNew = hdfs://sandbox.hortonworks.com:8020/user/oozie/share/lib/lib_20170728140342
        status = Successful
        
        
[oozie@sandbox ~]$ oozie admin -shareliblist
[Available ShareLib]
hive
distcp
mapreduce-streaming
spark
oozie
hcatalog
hive2
sqoop
pig
spark_orig


[oozie@sandbox ~]$

This appears to be an improvement, but restarting oozie was unsuccessful

[#5]

@Dave Welden

This is related to the sharelib service that was looking for sharelib folder on your local file system instead of my hdfs.

So to fix it:

stop oozie

edit conf/oozie-site.xml

<property>        
      <name>oozie.service.HadoopAccessorService.hadoop.configurations</name> 
      <value>*=/usr/local/hadoop/etc/hadoop/</value>
</property>

Restart oozie.

[#6]

The original property value was *=/etc/hadoop/conf. I changed it to *=/usr/local/hadoop/etc/hadoop/ and attempted restart of oozie. Service still will not start.

[oozie@sandbox ~]$ ll /usr/local/hadoop/etc/hadoop
ls: cannot access /usr/local/hadoop/etc/hadoop: No such file or directory
[oozie@sandbox ~]$ ll /etc/hadoop/conf
lrwxrwxrwx 1 root root 35 Jul 28 13:42 /etc/hadoop/conf -> /usr/hdp/current/hadoop-client/conf
[oozie@sandbox ~]$

[#7]

@Dave Welden

Can you upload these files ? Do you have any errors in oozie-error.log and catalina.out in /var/log/oozie.

Can you revert the oozie-site.xml values from *=/usr/local/hadoop/etc/hadoop/ to *=/etc/hadoop/conf

[#8]

@Geoffrey Shelton Okot

First let me say I truly appreciate your time and efforts to help a noobie out.

I scrapped the sandbox I had and started over with a fresh docker install (which defaults to the *=/etc/hadoop/conf setting in oozie-site.xml). After the NiFi install, a message came up to restart any services with the Yellow recycle icon. From the Dashboard screen I clicked Actions and selected Restart All Required. It abended on the oozie restart with the following error (several lines in the call stack omitted for brevity):

  File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py", line 159, in <module>
    OozieServer().execute()
...
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh cp /usr/hdp/current/falcon-client/oozie/ext/falcon-oozie-el-extension-*.jar /usr/hdp/current/oozie-server/libext' returned 1. cp: `/usr/hdp/current/falcon-client/oozie/ext/falcon-oozie-el-extension-0.10.0.2.6.1.0-129.jar' and `/usr/hdp/current/oozie-server/libext/falcon-oozie-el-extension-0.10.0.2.6.1.0-129.jar' are the same file

A copy error caused by a duplicate/existing jar file seems to be the culprit.

Files you requested: davewelden145801.zip

[#9]

@Geoffrey Shelton Okot

I renamed the duplicate JAR file and the restart of oozie was successful. I still have an issue with Atlas not coming up, but it is not required for this tutorial. I will look into resolving it at a later time.

Again, thank you for your assistance in troubleshooting.