Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

CDH 5.4.4 Upload Oozie ShareLib - Command timed-out after 270 second

CDH 5.4.4 Upload Oozie ShareLib - Command timed-out after 270 second

New Contributor

i found a bug while activating the 5.4.4  parcels, during the upgrade the oozie gets timed out after 270seconds, (may be my vm cluster is slow), after a bit of soul searching i found a way to get around the issue by issueing manual command to do the upload of the sharelib to the hdfs directory.  it was taking nearly 5min on my slow cluster - (one of my disk was burst on raid5 -so writes are quite slow).

 

when i tried repeatedly from CM it always timed out after 5min, uploads about 220 jar files,  total there are 318 files in that folder to upload to the /user/oozie/share/lib directory.

 

here is the maual work around ,  with gpvm209 being the namenode example - 

 

# cd /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/bin

 

# ./oozie-setup sharelib create -fs hdfs://gpvm209.geever.local:8020 -locallib /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/oozie/oozie-sharelib-yarn
setting JAVA_LIBRARY_PATH="$JAVA_LIBRARY_PATH:/usr/lib/hadoop/lib/native"
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting OOZIE_HTTPS_PORT=11443
setting OOZIE_HTTPS_KEYSTORE_PASS=password
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.port=${OOZIE_HTTPS_PORT}"
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.keystore.pass=${OOZIE_HTTPS_KEYSTORE_PASS}"
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
setting JAVA_LIBRARY_PATH="$JAVA_LIBRARY_PATH:/usr/lib/hadoop/lib/native"
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting OOZIE_HTTPS_PORT=11443
setting OOZIE_HTTPS_KEYSTORE_PASS=password
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.port=${OOZIE_HTTPS_PORT}"
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.keystore.pass=${OOZIE_HTTPS_KEYSTORE_PASS}"
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
the destination path for sharelib is: /user/oozie/share/lib/lib_20150711183547

 

[root@gpvm00 bin]#

[root@gpvm00 ~]# hadoop fs -ls -R /user/oozie/share/lib |wc -l
318

 

 

works like a champ, and now I can run oozie happyly.

 

 

ta

Geever Paul

 

 

5 REPLIES 5

Re: CDH 5.4.4 Upload Oozie ShareLib - Command timed-out after 270 seconds

Do you happen to be running Cloudera Manager 5.3.x or earlier?

Regards,
Gautam Gopalakrishnan

Re: CDH 5.4.4 Upload Oozie ShareLib - Command timed-out after 270 seconds

New Contributor

Hi

 

I was pretty sure i was a point release on 5.2.x , the activate process also gone through the "upgrade" and towwards end of it got aborted due to timeout.

 

I kept retry, delete add and many other options before realise it's timing out - before comlete the sharelib jar uploads appox 300+.

 

Regards 

Geever Paul

Re: CDH 5.4.4 Upload Oozie ShareLib - Command timed-out after 270 seconds

Did you mean you were using CM 5.2 to manage CDH5.4? If so this not
supported. Please upgrade to CM 5.4 when possible.

Regards,
Gautam Gopalakrishnan

Re: CDH 5.4.4 Upload Oozie ShareLib - Command timed-out after 270 seconds

New Contributor

Hi Gautam

 

I am running 5.4.1 CM, it's the CDH that was older which got upgraded from 5.2.x  to 5.4.4 where I encountered the slow jar upload and time out issue.,

 

everything else seems working okay, 

Thanks again.

Geever Paul

Highlighted

Re: CDH 5.4.4 Upload Oozie ShareLib - Command timed-out after 270 seconds

Explorer
I did install from latest a couple of weeks ago, haven't seen this error. Needed to reinstall from scratch and now facing this 270 seconds time out.
Attempted to follow provided manual step but command fails:
Error: E0024: Oozie configuration directory does not exist [/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/oozie/conf]

Stack trace for the error was (for debug purposes):
--------------------------------------
org.apache.oozie.service.ServiceException: E0024: Oozie configuration directory does not exist [/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/oozie/conf]
at org.apache.oozie.service.ConfigurationService.getConfigurationDirectory(ConfigurationService.java:175)
at org.apache.oozie.service.XLogService.init(XLogService.java:138)
at org.apache.oozie.service.Services.setServiceInternal(Services.java:386)
at org.apache.oozie.service.Services.<init>(Services.java:111)
at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:142)
at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:57)

I do see oozie jar files under
/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/oozie/oozie-sharelib-yarn/lib/oozie/

Granted, it has been long time and many releases,
Has command syntax changed?