Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Restart required for Atlas and Oozie after install of NiFi

Highlighted

Restart required for Atlas and Oozie after install of NiFi

Contributor

After installing NiFi, Oozie Server and Atlas Metadata Server components require a restart. Restart is failing. oozie-error.log has the following:

ERROR ShareLibService:517 ... E0104: Could not fully initiLize service ShareLibService, Not able to cache sharelib. An Admin needs to install the sharelib with oozie-setup.sh and issue the 'oozie admin' CLI command to update the sharelib
8 REPLIES 8

Re: Restart required for Atlas and Oozie after install of NiFi

Contributor

However, it appears that a sharelib already exists as revealed by

hdfs dfs -ls /user/oozie/share/lib
Found 1 items
drwxr-xr-x   - oozie hdfs          0 2017-07-28 14:06 /user/oozie/share/lib/lib_20170728140342

Re: Restart required for Atlas and Oozie after install of NiFi

Mentor

@Dave Welden

It seems your oozie is picking old libraries can you do the following

As root

# su - oozie

Check the sharedlibs below is output from my cluster

$ oozie admin -shareliblist 
[Available ShareLib] 
hive 
distcp 
mapreduce-streaming 
spark oozie 
hcatalog 
hive2 
sqoop 
pig 
spark_orig 

Run the update

$ oozie admin -sharelibupdate 
[ShareLib update status] 
sharelibDirOld = hdfs://test.com:8020/user/oozie/share/lib/lib_20170731003540 
host = http://test.com:11000/oozie 
sharelibDirNew = hdfs://test.com:8020/user/oozie/share/lib/lib_20170731003540 
status = Successful

Then retry start the oozie

Re: Restart required for Atlas and Oozie after install of NiFi

Contributor

Hi @Geoffrey Shelton Okot

I tried the above as follows:

sandbox login: root
root@sandbox.hortonworks.com's password:
[root@sandbox ~]# su - oozie
[oozie@sandbox ~]$ oozie admin -shareliblist
[Available ShareLib]


[oozie@sandbox ~]$ oozie admin -sharelibupdate
[ShareLib update status]
        sharelibDirOld = null
        host = http://sandbox.hortonworks.com:11000/oozie
        sharelibDirNew = hdfs://sandbox.hortonworks.com:8020/user/oozie/share/lib/lib_20170728140342
        status = Successful
        
        
[oozie@sandbox ~]$ oozie admin -shareliblist
[Available ShareLib]
hive
distcp
mapreduce-streaming
spark
oozie
hcatalog
hive2
sqoop
pig
spark_orig


[oozie@sandbox ~]$

This appears to be an improvement, but restarting oozie was unsuccessful

Re: Restart required for Atlas and Oozie after install of NiFi

Mentor

@Dave Welden

This is related to the sharelib service that was looking for sharelib folder on your local file system instead of my hdfs.

So to fix it:

stop oozie

edit conf/oozie-site.xml

<property>        
      <name>oozie.service.HadoopAccessorService.hadoop.configurations</name> 
      <value>*=/usr/local/hadoop/etc/hadoop/</value>
</property>

Restart oozie.

Re: Restart required for Atlas and Oozie after install of NiFi

Contributor

The original property value was *=/etc/hadoop/conf. I changed it to *=/usr/local/hadoop/etc/hadoop/ and attempted restart of oozie. Service still will not start.

[oozie@sandbox ~]$ ll /usr/local/hadoop/etc/hadoop
ls: cannot access /usr/local/hadoop/etc/hadoop: No such file or directory
[oozie@sandbox ~]$ ll /etc/hadoop/conf lrwxrwxrwx 1 root root 35 Jul 28 13:42 /etc/hadoop/conf -> /usr/hdp/current/hadoop-client/conf
[oozie@sandbox ~]$

Re: Restart required for Atlas and Oozie after install of NiFi

Mentor

@Dave Welden

Can you upload these files ? Do you have any errors in oozie-error.log and catalina.out in /var/log/oozie.

Can you revert the oozie-site.xml values from *=/usr/local/hadoop/etc/hadoop/ to *=/etc/hadoop/conf



Re: Restart required for Atlas and Oozie after install of NiFi

Contributor

@Geoffrey Shelton Okot

First let me say I truly appreciate your time and efforts to help a noobie out.

I scrapped the sandbox I had and started over with a fresh docker install (which defaults to the *=/etc/hadoop/conf setting in oozie-site.xml). After the NiFi install, a message came up to restart any services with the Yellow recycle icon. From the Dashboard screen I clicked Actions and selected Restart All Required. It abended on the oozie restart with the following error (several lines in the call stack omitted for brevity):

  File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py", line 159, in <module>
    OozieServer().execute()
...
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'ambari-sudo.sh cp /usr/hdp/current/falcon-client/oozie/ext/falcon-oozie-el-extension-*.jar /usr/hdp/current/oozie-server/libext' returned 1. cp: `/usr/hdp/current/falcon-client/oozie/ext/falcon-oozie-el-extension-0.10.0.2.6.1.0-129.jar' and `/usr/hdp/current/oozie-server/libext/falcon-oozie-el-extension-0.10.0.2.6.1.0-129.jar' are the same file

A copy error caused by a duplicate/existing jar file seems to be the culprit.

Files you requested: davewelden145801.zip

Re: Restart required for Atlas and Oozie after install of NiFi

Contributor

@Geoffrey Shelton Okot

I renamed the duplicate JAR file and the restart of oozie was successful. I still have an issue with Atlas not coming up, but it is not required for this tutorial. I will look into resolving it at a later time.

Again, thank you for your assistance in troubleshooting.

Don't have an account?
Coming from Hortonworks? Activate your account here