Support Questions

spin0 · ‎12-21-2016

I just install sqoop2 server on a node (Via Cloudera Manager) and the service could not start (failed to start).

Not sure the issue. I see some key word in the stderr and stdout logs.

I use CDH-5.9.0-1.

Need helps.

Thank you.

stdout log

Wed Dec 21 14:52:05 PST 2016
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
using 5 as CDH_VERSION
log4j:ERROR A "org.apache.log4j.RollingFileAppender" object is not assignable to a "org.apache.log4j.Appender" variable.
log4j:ERROR The class "org.apache.log4j.Appender" was loaded by 
log4j:ERROR [org.apache.catalina.loader.StandardClassLoader@2d33d68a] whereas object of type 
log4j:ERROR "org.apache.log4j.RollingFileAppender" was loaded by [WebappClassLoader
  context: /sqoop
  delegate: false
  repositories:
    /WEB-INF/classes/
----------> Parent Classloader:
org.apache.catalina.loader.StandardClassLoader@2d33d68a
].
log4j:ERROR Could not instantiate appender named "RFA".
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.NativeCodeLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Wed Dec 21 14:52:09 PST 2016
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
using 5 as CDH_VERSION
log4j:ERROR A "org.apache.log4j.RollingFileAppender" object is not assignable to a "org.apache.log4j.Appender" variable.
log4j:ERROR The class "org.apache.log4j.Appender" was loaded by 
log4j:ERROR [org.apache.catalina.loader.StandardClassLoader@3e4523d6] whereas object of type 
log4j:ERROR "org.apache.log4j.RollingFileAppender" was loaded by [WebappClassLoader
  context: /sqoop
  delegate: false
  repositories:
    /WEB-INF/classes/
----------> Parent Classloader:
org.apache.catalina.loader.StandardClassLoader@3e4523d6
].
log4j:ERROR Could not instantiate appender named "RFA".
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.NativeCodeLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Wed Dec 21 14:52:15 PST 2016
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
using 5 as CDH_VERSION
log4j:ERROR A "org.apache.log4j.RollingFileAppender" object is not assignable to a "org.apache.log4j.Appender" variable.
log4j:ERROR The class "org.apache.log4j.Appender" was loaded by 
log4j:ERROR [org.apache.catalina.loader.StandardClassLoader@3e4523d6] whereas object of type 
log4j:ERROR "org.apache.log4j.RollingFileAppender" was loaded by [WebappClassLoader
  context: /sqoop
  delegate: false
  repositories:
    /WEB-INF/classes/
----------> Parent Classloader:
org.apache.catalina.loader.StandardClassLoader@3e4523d6
].
log4j:ERROR Could not instantiate appender named "RFA".
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.NativeCodeLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Wed Dec 21 14:52:22 PST 2016
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
using 5 as CDH_VERSION
log4j:ERROR A "org.apache.log4j.RollingFileAppender" object is not assignable to a "org.apache.log4j.Appender" variable.
log4j:ERROR The class "org.apache.log4j.Appender" was loaded by 
log4j:ERROR [org.apache.catalina.loader.StandardClassLoader@264a1535] whereas object of type 
log4j:ERROR "org.apache.log4j.RollingFileAppender" was loaded by [WebappClassLoader
  context: /sqoop
  delegate: false
  repositories:
    /WEB-INF/classes/
----------> Parent Classloader:
org.apache.catalina.loader.StandardClassLoader@264a1535
].
log4j:ERROR Could not instantiate appender named "RFA".
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.NativeCodeLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

stderr log

Wed Dec 21 14:52:22 PST 2016
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/meta/cdh_env.sh ']'
+ OLD_IFS=' 	
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS=' 	
'
+ COUNT=1
++ seq 1 1
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.9.0-1.cdh5.9.0.p0.23
+ . /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.9.0-1.cdh5.9.0.p0.23
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hadoop-kms
++ export CDH_PARQUET_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/parquet
++ CDH_PARQUET_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/parquet
++ export CDH_AVRO_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/avro
++ CDH_AVRO_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/avro
+ locate_cdh_java_home
+ '[' -z '' ']'
+ '[' -z /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/bigtop-utils ']'
+ local BIGTOP_DETECT_JAVAHOME=
+ for candidate in '"${JSVC_HOME}"' '"${JSVC_HOME}/.."' '"/usr/lib/bigtop-utils"' '"/usr/libexec"'
+ '[' -e /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/bigtop-utils/bigtop-detect-javahome ']'
+ BIGTOP_DETECT_JAVAHOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/bigtop-utils/bigtop-detect-javahome
+ break
+ '[' -z /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/bigtop-utils/bigtop-detect-javahome ']'
+ . /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/bigtop-utils/bigtop-detect-javahome
++ BIGTOP_DEFAULTS_DIR=/etc/default
++ '[' -n /etc/default -a -r /etc/default/bigtop-utils ']'
++ JAVA6_HOME_CANDIDATES=('/usr/lib/j2sdk1.6-sun' '/usr/lib/jvm/java-6-sun' '/usr/lib/jvm/java-1.6.0-sun-1.6.0' '/usr/lib/jvm/j2sdk1.6-oracle' '/usr/lib/jvm/j2sdk1.6-oracle/jre' '/usr/java/jdk1.6' '/usr/java/jre1.6')
++ OPENJAVA6_HOME_CANDIDATES=('/usr/lib/jvm/java-1.6.0-openjdk' '/usr/lib/jvm/jre-1.6.0-openjdk')
++ JAVA7_HOME_CANDIDATES=('/usr/java/jdk1.7' '/usr/java/jre1.7' '/usr/lib/jvm/j2sdk1.7-oracle' '/usr/lib/jvm/j2sdk1.7-oracle/jre' '/usr/lib/jvm/java-7-oracle')
++ OPENJAVA7_HOME_CANDIDATES=('/usr/lib/jvm/java-1.7.0-openjdk' '/usr/lib/jvm/java-7-openjdk')
++ JAVA8_HOME_CANDIDATES=('/usr/java/jdk1.8' '/usr/java/jre1.8' '/usr/lib/jvm/j2sdk1.8-oracle' '/usr/lib/jvm/j2sdk1.8-oracle/jre' '/usr/lib/jvm/java-8-oracle')
++ OPENJAVA8_HOME_CANDIDATES=('/usr/lib/jvm/java-1.8.0-openjdk' '/usr/lib/jvm/java-8-openjdk')
++ MISCJAVA_HOME_CANDIDATES=('/Library/Java/Home' '/usr/java/default' '/usr/lib/jvm/default-java' '/usr/lib/jvm/java-openjdk' '/usr/lib/jvm/jre-openjdk')
++ case ${BIGTOP_JAVA_MAJOR} in
++ JAVA_HOME_CANDIDATES=(${JAVA7_HOME_CANDIDATES[@]} ${JAVA8_HOME_CANDIDATES[@]} ${MISCJAVA_HOME_CANDIDATES[@]} ${OPENJAVA7_HOME_CANDIDATES[@]} ${OPENJAVA8_HOME_CANDIDATES[@]})
++ '[' -z '' ']'
++ for candidate_regex in '${JAVA_HOME_CANDIDATES[@]}'
+++ ls -rvd /usr/java/jdk1.7.0_67-cloudera
++ for candidate in '`ls -rvd ${candidate_regex}* 2>/dev/null`'
++ '[' -e /usr/java/jdk1.7.0_67-cloudera/bin/java ']'
++ export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
++ JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
++ break 2
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_67-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
+ set -x
+ echo 'using 5 as CDH_VERSION'
+ replace_conf_dir
+ find /run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER -type f '!' -path '/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/logs/*' '!' -name '*.log' '!' -name '*.keytab' '!' -name '*jceks' -exec perl -pi -e 's#{{CMF_CONF_DIR}}#/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER#g' '{}' ';'
Can't open /run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/supervisor.conf: Permission denied.
+ perl -pi -e 's#{{CMF_CONF_DIR}}#/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER#g' /run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/catalina_logging.properties /run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/cloudera-monitor.properties /run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/cloudera-stack-monitor.properties /run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/log4j.properties /run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/sqoop_bootstrap.properties /run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/sqoop.properties
+ '[' 5 -ge 5 ']'
+ '[' -z /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/meta/cdh_env.sh ']'
+ TOMCAT_CONF=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/sqoop2/../../etc/sqoop2/tomcat-conf.dist
+ export CATALINA_BASE=/var/lib/sqoop2/tomcat-deployment
+ CATALINA_BASE=/var/lib/sqoop2/tomcat-deployment
+ env TOMCAT_CONF=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/sqoop2/../../etc/sqoop2/tomcat-conf.dist TOMCAT_DEPLOYMENT=/var/lib/sqoop2/tomcat-deployment SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/sqoop2 bash /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/sqoop2/tomcat-deployment.sh
+ export CATALINA_TMPDIR=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/temp
+ CATALINA_TMPDIR=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/temp
+ export CATALINA_OUT=/var/log/sqoop2/sqoop.out
+ CATALINA_OUT=/var/log/sqoop2/sqoop.out
+ export CATALINA_PID=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/sqoop.pid
+ CATALINA_PID=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/sqoop.pid
++ replace_pid -Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError '-XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid{{PID}}.hprof' -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
++ echo -Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError '-XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid{{PID}}.hprof' -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
++ sed 's#{{PID}}#12585#g'
+ SQOOP_SERVER_OPTS='-Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid12585.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh'
+ OPTIONS='-Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid12585.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh'
+ OPTIONS='-Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid12585.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/log4j.properties'
+ OPTIONS='-Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid12585.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/log4j.properties -Dsqoop.config.dir=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER'
+ OPTIONS='-Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid12585.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/log4j.properties -Dsqoop.config.dir=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER -Djava.net.preferIPv4Stack=true'
+ OPTIONS='-Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid12585.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/log4j.properties -Dsqoop.config.dir=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000'
+ OPTIONS='-Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid12585.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/log4j.properties -Dsqoop.config.dir=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005'
++ replace_pid
++ echo
++ sed 's#{{PID}}#12585#g'
+ CATALINA_OPTS=
+ export 'CATALINA_OPTS=-Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid12585.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/log4j.properties -Dsqoop.config.dir=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005 '
+ CATALINA_OPTS='-Xms318767104 -Xmx318767104 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/sqoop_sqoop-SQOOP_SERVER-7fbdaee63bd87efee4217fb2ae176aef_pid12585.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/log4j.properties -Dsqoop.config.dir=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005 '
+ export LOGGING_CONFIG=-Djava.util.logging.config.file=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/catalina_logging.properties
+ LOGGING_CONFIG=-Djava.util.logging.config.file=/run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/catalina_logging.properties
+ mkdir -p /run/cloudera-scm-agent/process/227-sqoop-SQOOP_SERVER/temp
+ EXT_LIBS_DIR=/var/lib/sqoop2
+ cp /usr/share/java/mysql-connector-java.jar /var/lib/sqoop2
+ cp /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar /var/lib/sqoop2
+ cp /usr/share/java/oracle-connector-java.jar /var/lib/sqoop2
+ '[' '' == upgrade -o '' == create-database ']'
+ exec /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/bigtop-tomcat/bin/catalina.sh run
Dec 21, 2016 2:52:23 PM org.apache.catalina.startup.ClassLoaderFactory validateFile
WARNING: Problem with directory [/usr/share/java], exists: [false], isDirectory: [false], canRead: [false]
Dec 21, 2016 2:52:24 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
Dec 21, 2016 2:52:24 PM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-12000
Dec 21, 2016 2:52:24 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 795 ms
Dec 21, 2016 2:52:24 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Dec 21, 2016 2:52:24 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.45
Dec 21, 2016 2:52:24 PM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive sqoop.war

spin0 · ‎12-22-2016

I have additional info. regrading the error message in red (Role log).

Note: This is a new installation of sqoop2 using Cloudera Manager and it uses postgresql (Instead of Derby database).

I'm not sure why this error after the installation.

I take a look in the following directires and they have files for postgresql, it seems to be OK for postgresql.

r--r-- sqoop2 sqoop2 /var/lib/sqoop2/postgresql-9.0-801.jdbc4.jar

--r-- root root /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/jars/postgresql-9.0-801.jdbc4.jar

-xr-x root root /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/jars/postgresql-9.1-901.jdbc4.jar

--r-- root root /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/jars/post.jar

lrwxrwxrwx root root /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/sqoop2/webapps/sqoop/WEB-INF/lib/postgresql-9.1-901/../../../jars/postgresql-9.1-901.jdbc4.jar

Any helps will be appraciated.

Role log

Sqoop server failed to start

java.lang.RuntimeException: Failure in server initialization

at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:63)

at org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:41)

at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4236)

at org.apache.catalina.core.StandardContext.start(StandardContext.java:4739)

at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)

at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780)

at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)

at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:944)

at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:779)

at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:505)

at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322)

at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325)

at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)

at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069)

at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)

at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061)

at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)

at org.apache.catalina.core.StandardService.start(StandardService.java:525)

at org.apache.catalina.core.StandardServer.start(StandardServer.java:759)

at org.apache.catalina.startup.Catalina.start(Catalina.java:595)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)

at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)

Caused by: org.apache.sqoop.common.SqoopException: JDBCREPO_0007:Unable to lease link

at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:64)

at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:87)

at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:63)

at org.apache.sqoop.repository.JdbcRepository.isRepositorySuitableForUse(JdbcRepository.java:144)

at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:129)

at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:78)

at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:56)

... 25 more

Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:136)

at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)

at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)

at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)

at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)

at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)

at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)

at org.postgresql.Driver.makeConnection(Driver.java:393)

at org.postgresql.Driver.connect(Driver.java:267)

at java.sql.DriverManager.getConnection(DriverManager.java:571)

at java.sql.DriverManager.getConnection(DriverManager.java:187)

at org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:78)

at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)

at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148)

at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)

at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:62)

... 31 more

Caused by: java.net.ConnectException: Connection refused

at java.net.PlainSocketImpl.socketConnect(Native Method)

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

at java.net.Socket.connect(Socket.java:579)

at java.net.Socket.connect(Socket.java:528)

at java.net.Socket.<init>(Socket.java:425)

at java.net.Socket.<init>(Socket.java:208)

at org.postgresql.core.PGStream.<init>(PGStream.java:62)

at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76)

View solution in original post

spin0 · ‎02-02-2017

I finally found the issue and resolved it.

Note: This is with CDH-5.9.0-1 and installation was executed from Cloudera Manager (fresh installation).

Issue:

After adding Sqoop2 service from Cloudera Manager, Sqoop service cannot be started.

Cause:

Base on th error in Role log, there is an error that was bugging me the most.

"Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

"

Basically during the installation of Sqoop2 service, the script that does the installation somehow did not create the DB repository (in this case it was PostgreSQL).

As you can see the error above it is obvious that was something relating to back-end.

Looking into Sqoop configuration from Clouder Manager, the PostgreSQL database name was sqoop. The DB account was sa and DB Account password was empty. It is very strange.

Solution:

1- You need to access to PostgreSQL to find out whether sqoop account and database are present which they are not. In the case, sa account. And sure enough it was not present. It rings the bell right the way.

Changing Embedded PostgreSQL Database Passwords

https://www.cloudera.com/documentation/enterprise/5-6-x/topics/cm_ig_embed_pstgrs.html

2- Create account (role) and database for sqoop.

Configuring Sqoop 2 to Use PostgreSQLing Embedded

https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cdh_ig_sqoop2_configure.html

In this case I created account: sqoop and its password. And database name is sqoop.

3- You need to upgrade sqoop database to make sure that all structures are correct.

Use Cloudera Manager to do Sqoop database upgrade.

Note: When you creat a new database, there is not structure for Sqoop. By doing the upgrade, the system adjusts all Sqoop tabales needed.

4- Configure new account, database, host and port using Cloudera Manager. Save it.

Use Cloudera Manager to configure Sqoop database info.

- Sqoop repository DB server with port (Default 7432 for PostgreSQL)

Sqoop Repository Database Host:

- Sqoop database name

Sqoop Repository Database Name:

- DB user account

Sqoop Repository Database User:

- DB password

Sqoop Repository Database Password:

5- Start Sqoop service

View solution in original post

spin0 · ‎12-22-2016