Welcome to the Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this topic

CDH 5.4.4 Upload Oozie ShareLib - Command timed-out after 270 second

avatar
New Contributor

i found a bug while activating the 5.4.4  parcels, during the upgrade the oozie gets timed out after 270seconds, (may be my vm cluster is slow), after a bit of soul searching i found a way to get around the issue by issueing manual command to do the upload of the sharelib to the hdfs directory.  it was taking nearly 5min on my slow cluster - (one of my disk was burst on raid5 -so writes are quite slow).

 

when i tried repeatedly from CM it always timed out after 5min, uploads about 220 jar files,  total there are 318 files in that folder to upload to the /user/oozie/share/lib directory.

 

here is the maual work around ,  with gpvm209 being the namenode example - 

 

# cd /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/bin

 

# ./oozie-setup sharelib create -fs hdfs://gpvm209.geever.local:8020 -locallib /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/oozie/oozie-sharelib-yarn
setting JAVA_LIBRARY_PATH="$JAVA_LIBRARY_PATH:/usr/lib/hadoop/lib/native"
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting OOZIE_HTTPS_PORT=11443
setting OOZIE_HTTPS_KEYSTORE_PASS=password
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.port=${OOZIE_HTTPS_PORT}"
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.keystore.pass=${OOZIE_HTTPS_KEYSTORE_PASS}"
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
setting JAVA_LIBRARY_PATH="$JAVA_LIBRARY_PATH:/usr/lib/hadoop/lib/native"
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/var/lib/oozie/tomcat-deployment
setting OOZIE_HTTPS_PORT=11443
setting OOZIE_HTTPS_KEYSTORE_PASS=password
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.port=${OOZIE_HTTPS_PORT}"
setting CATALINA_OPTS="$CATALINA_OPTS -Doozie.https.keystore.pass=${OOZIE_HTTPS_KEYSTORE_PASS}"
setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m"
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_LOG=/var/log/oozie
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/jars/slf4j-simple-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
the destination path for sharelib is: /user/oozie/share/lib/lib_20150711183547

 

[root@gpvm00 bin]#

[root@gpvm00 ~]# hadoop fs -ls -R /user/oozie/share/lib |wc -l
318

 

 

works like a champ, and now I can run oozie happyly.

 

 

ta

Geever Paul

 

 

Who agreed with this topic