After installing NiFi, Oozie Server and Atlas Metadata Server components require a restart. Restart is failing. oozie-error.log has the following:
ERROR ShareLibService:517 ... E0104: Could not fully initiLize service ShareLibService, Not able to cache sharelib. An Admin needs to install the sharelib with oozie-setup.sh and issue the 'oozie admin' CLI command to update the sharelib
However, it appears that a sharelib already exists as revealed by
hdfs dfs -ls /user/oozie/share/lib Found 1 items drwxr-xr-x - oozie hdfs 0 2017-07-28 14:06 /user/oozie/share/lib/lib_20170728140342
It seems your oozie is picking old libraries can you do the following
# su - oozie
Check the sharedlibs below is output from my cluster
$ oozie admin -shareliblist [Available ShareLib] hive distcp mapreduce-streaming spark oozie hcatalog hive2 sqoop pig spark_orig
Run the update
$ oozie admin -sharelibupdate [ShareLib update status] sharelibDirOld = hdfs://test.com:8020/user/oozie/share/lib/lib_20170731003540 host = http://test.com:11000/oozie sharelibDirNew = hdfs://test.com:8020/user/oozie/share/lib/lib_20170731003540 status = Successful
Then retry start the oozie
Hi @Geoffrey Shelton Okot
I tried the above as follows:
sandbox login: root email@example.com's password: [root@sandbox ~]# su - oozie [oozie@sandbox ~]$ oozie admin -shareliblist [Available ShareLib] [oozie@sandbox ~]$ oozie admin -sharelibupdate [ShareLib update status] sharelibDirOld = null host = http://sandbox.hortonworks.com:11000/oozie sharelibDirNew = hdfs://sandbox.hortonworks.com:8020/user/oozie/share/lib/lib_20170728140342 status = Successful [oozie@sandbox ~]$ oozie admin -shareliblist [Available ShareLib] hive distcp mapreduce-streaming spark oozie hcatalog hive2 sqoop pig spark_orig [oozie@sandbox ~]$
This appears to be an improvement, but restarting oozie was unsuccessful
This is related to the sharelib service that was looking for sharelib folder on your local file system instead of my hdfs.
So to fix it:
<property> <name>oozie.service.HadoopAccessorService.hadoop.configurations</name> <value>*=/usr/local/hadoop/etc/hadoop/</value> </property>
The original property value was *=/etc/hadoop/conf. I changed it to *=/usr/local/hadoop/etc/hadoop/ and attempted restart of oozie. Service still will not start.
[oozie@sandbox ~]$ ll /usr/local/hadoop/etc/hadoop
ls: cannot access /usr/local/hadoop/etc/hadoop: No such file or directory
[oozie@sandbox ~]$ ll /etc/hadoop/conf lrwxrwxrwx 1 root root 35 Jul 28 13:42 /etc/hadoop/conf -> /usr/hdp/current/hadoop-client/conf
Can you upload these files ? Do you have any errors in oozie-error.log and catalina.out in /var/log/oozie.
Can you revert the oozie-site.xml values from *=/usr/local/hadoop/etc/hadoop/ to *=/etc/hadoop/conf
@Geoffrey Shelton Okot
First let me say I truly appreciate your time and efforts to help a noobie out.
I scrapped the sandbox I had and started over with a fresh docker install (which defaults to the *=/etc/hadoop/conf setting in oozie-site.xml). After the NiFi install, a message came up to restart any services with the Yellow recycle icon. From the Dashboard screen I clicked Actions and selected Restart All Required. It abended on the oozie restart with the following error (several lines in the call stack omitted for brevity):
File "/var/lib/ambari-agent/cache/common-services/OOZIE/220.127.116.11.0/package/scripts/oozie_server.py", line 159, in <module> OozieServer().execute() ... raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh cp /usr/hdp/current/falcon-client/oozie/ext/falcon-oozie-el-extension-*.jar /usr/hdp/current/oozie-server/libext' returned 1. cp: `/usr/hdp/current/falcon-client/oozie/ext/falcon-oozie-el-extension-0.10.0.2.6.1.0-129.jar' and `/usr/hdp/current/oozie-server/libext/falcon-oozie-el-extension-0.10.0.2.6.1.0-129.jar' are the same file
A copy error caused by a duplicate/existing jar file seems to be the culprit.
Files you requested: davewelden145801.zip
@Geoffrey Shelton Okot
I renamed the duplicate JAR file and the restart of oozie was successful. I still have an issue with Atlas not coming up, but it is not required for this tutorial. I will look into resolving it at a later time.
Again, thank you for your assistance in troubleshooting.