Created 10-31-2017 07:39 PM
After installing NiFi, Oozie Server and Atlas Metadata Server components require a restart. Restart is failing. oozie-error.log has the following:
ERROR ShareLibService:517 ... E0104: Could not fully initiLize service ShareLibService, Not able to cache sharelib. An Admin needs to install the sharelib with oozie-setup.sh and issue the 'oozie admin' CLI command to update the sharelib
Created 10-31-2017 07:43 PM
However, it appears that a sharelib already exists as revealed by
hdfs dfs -ls /user/oozie/share/lib Found 1 items drwxr-xr-x - oozie hdfs 0 2017-07-28 14:06 /user/oozie/share/lib/lib_20170728140342
Created 10-31-2017 08:55 PM
It seems your oozie is picking old libraries can you do the following
As root
# su - oozie
Check the sharedlibs below is output from my cluster
$ oozie admin -shareliblist [Available ShareLib] hive distcp mapreduce-streaming spark oozie hcatalog hive2 sqoop pig spark_orig
Run the update
$ oozie admin -sharelibupdate [ShareLib update status] sharelibDirOld = hdfs://test.com:8020/user/oozie/share/lib/lib_20170731003540 host = http://test.com:11000/oozie sharelibDirNew = hdfs://test.com:8020/user/oozie/share/lib/lib_20170731003540 status = Successful
Then retry start the oozie
Created 11-01-2017 01:19 PM
Hi @Geoffrey Shelton Okot
I tried the above as follows:
sandbox login: root root@sandbox.hortonworks.com's password: [root@sandbox ~]# su - oozie [oozie@sandbox ~]$ oozie admin -shareliblist [Available ShareLib] [oozie@sandbox ~]$ oozie admin -sharelibupdate [ShareLib update status] sharelibDirOld = null host = http://sandbox.hortonworks.com:11000/oozie sharelibDirNew = hdfs://sandbox.hortonworks.com:8020/user/oozie/share/lib/lib_20170728140342 status = Successful [oozie@sandbox ~]$ oozie admin -shareliblist [Available ShareLib] hive distcp mapreduce-streaming spark oozie hcatalog hive2 sqoop pig spark_orig [oozie@sandbox ~]$
This appears to be an improvement, but restarting oozie was unsuccessful
Created 10-31-2017 09:13 PM
This is related to the sharelib service that was looking for sharelib folder on your local file system instead of my hdfs.
So to fix it:
stop oozie
edit conf/oozie-site.xml
<property> <name>oozie.service.HadoopAccessorService.hadoop.configurations</name> <value>*=/usr/local/hadoop/etc/hadoop/</value> </property>
Restart oozie.
Created 11-01-2017 01:16 PM
The original property value was *=/etc/hadoop/conf. I changed it to *=/usr/local/hadoop/etc/hadoop/ and attempted restart of oozie. Service still will not start.
[oozie@sandbox ~]$ ll /usr/local/hadoop/etc/hadoop
ls: cannot access /usr/local/hadoop/etc/hadoop: No such file or directory
[oozie@sandbox ~]$ ll /etc/hadoop/conf lrwxrwxrwx 1 root root 35 Jul 28 13:42 /etc/hadoop/conf -> /usr/hdp/current/hadoop-client/conf
[oozie@sandbox ~]$
Created 11-01-2017 04:55 PM
Can you upload these files ? Do you have any errors in oozie-error.log and catalina.out in /var/log/oozie.
Can you revert the oozie-site.xml values from *=/usr/local/hadoop/etc/hadoop/ to *=/etc/hadoop/conf
Created 11-01-2017 05:53 PM
@Geoffrey Shelton Okot
First let me say I truly appreciate your time and efforts to help a noobie out.
I scrapped the sandbox I had and started over with a fresh docker install (which defaults to the *=/etc/hadoop/conf setting in oozie-site.xml). After the NiFi install, a message came up to restart any services with the Yellow recycle icon. From the Dashboard screen I clicked Actions and selected Restart All Required. It abended on the oozie restart with the following error (several lines in the call stack omitted for brevity):
File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py", line 159, in <module> OozieServer().execute() ... raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh cp /usr/hdp/current/falcon-client/oozie/ext/falcon-oozie-el-extension-*.jar /usr/hdp/current/oozie-server/libext' returned 1. cp: `/usr/hdp/current/falcon-client/oozie/ext/falcon-oozie-el-extension-0.10.0.2.6.1.0-129.jar' and `/usr/hdp/current/oozie-server/libext/falcon-oozie-el-extension-0.10.0.2.6.1.0-129.jar' are the same file
A copy error caused by a duplicate/existing jar file seems to be the culprit.
Files you requested: davewelden145801.zip
Created 11-01-2017 06:29 PM
@Geoffrey Shelton Okot
I renamed the duplicate JAR file and the restart of oozie was successful. I still have an issue with Atlas not coming up, but it is not required for this tutorial. I will look into resolving it at a later time.
Again, thank you for your assistance in troubleshooting.