question Re: CDH 5 fails to upload Oozie ShareLib on Ubuntu 12.04 in Archives of Support Questions (Read Only)

CDH 5 fails to upload Oozie ShareLib on Ubuntu 12.04

Steen P — Fri, 16 Sep 2022 09:06:05 GMT

I'm trying to install CDH 5 to a cluster I have, but it fails during installation with Cloudera Manager.

Specifically it's the installation of Oozie that are causing issues. The installer stops at the following step:

Failed to upload Oozie ShareLib.

Stderr

Failed to upload Oozie ShareLib.

Program: oozie/oozie.sh ["install-sharelib","oozie-sharelib-yarn.tar.gz","hdfs://SKP-Cluster-1L:8020"]

at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1986)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1982)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1980)

	at org.apache.hadoop.ipc.Client.call(Client.java:1409)
	at org.apache.hadoop.ipc.Client.call(Client.java:1362)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
	at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:362)
	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1438)
	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1260)
	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:525)
--------------------------------------

Stdout

Failed to upload Oozie ShareLib.

Program: oozie/oozie.sh ["install-sharelib","oozie-sharelib-yarn.tar.gz","hdfs://SKP-Cluster-1L:8020"]

Wed Aug 27 09:11:57 CEST 2014
JAVA_HOME=/usr/lib/jvm/java-7-oracle-cloudera
using 5 as CDH_VERSION
using /var/lib/oozie/tomcat-deployment as CATALINA_BASE
the destination path for sharelib is: /user/oozie/share/lib/lib_20140827091200

What could be causing this? And more important; how can I fix this?

Re: CDH 5 fails to upload Oozie ShareLib on Ubuntu 12.04

GautamG — Wed, 27 Aug 2014 08:55:35 GMT

Please have a look at the stderr and stdout logs that will be linked in the
Cloudera Manager view. Toward the end of those logs you will see the
reasons logged, please paste them here and we can work to fix the problem

Re: CDH 5 fails to upload Oozie ShareLib on Ubuntu 12.04

Steen P — Wed, 27 Aug 2014 09:27:53 GMT

So, when using "Links to full logs" it gives me this:

HTTP ERROR 502

Problem accessing /cmf/process/73/logs. Reason:

    Connection refused
Could not connect to host.

Powered by Jetty://

So I went in manually and did cat /var/log/oozie-cmf-oozie-OOZIE_SERVER-SKP-Cluster-1L.log.out and it gave me this:

http://pastebin.com/embed_js.php?i=yEHVcgMf

Re: CDH 5 fails to upload Oozie ShareLib on Ubuntu 12.04

Steen P — Thu, 04 Sep 2014 07:23:53 GMT

I ended up ditching Ubuntu and moved to CentOS instead. It solved my problem.

Re: CDH 5 fails to upload Oozie ShareLib on Ubuntu 12.04

pblaas — Thu, 04 Sep 2014 10:49:38 GMT

Hi,

Instead of creating a new thread i will continue on this one because im facing the same issue although I am on CentOS 6.

I performed a brand new installation with the CDH-5.1.2-1.cdh5.1.2.p0.3 Parcels, and a installation of the Cloudera Core packages.

I also recieved the same error in the oozie_server log like Steen P.

org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/oozie/share/lib/lib_20140904113544/oozie/json-simple-1.1.jar could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation.

Although the error is clear about the issue, im receiving this while performing a clean install.

Are you able to provide a fix or workarround for this?

Thanks.

Patrick

Re: CDH 5 fails to upload Oozie ShareLib on Ubuntu 12.04

pblaas — Thu, 04 Sep 2014 13:11:06 GMT

This weird error seemed to be caused by the fact the FQDN of the hosts where resolving to 127.0.0.1 aswell, which caused Hadoop to not start properly and the hadoop client unable to connect and upload the lib.

After rechecking all settings I was able to properly set up the parcel on the cluster.

Re: CDH 5 fails to upload Oozie ShareLib on Ubuntu 12.04

efxwill — Mon, 10 Nov 2014 15:33:45 GMT

Could you please provide a few details about the 127.0.0.1 issue? I'm seeing a similar problem in 5.2.0 and 5.1.3

Thanks Will Hanson