Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

Creating Sqoop 2 Database

avatar
author-rank DKumar
Contributor

Created on ‎12-24-2014 09:05 AM - edited ‎09-16-2022 02:16 AM

hi all,

 

when i try to install sqoop2, its giving an error.

 

Failed to execute command Create Sqoop Database on service Sqoop 2

 

Wed Dec 24 06:04:44 EST 2014
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/meta/cdh_env.sh ']'
+ OLD_IFS=' 	
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS=' 	
'
+ COUNT=1
++ seq 1 1
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.3.0-1.cdh5.3.0.p0.30
+ . /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.3.0-1.cdh5.3.0.p0.30
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-kms
+ locate_cdh_java_home
+ '[' -z '' ']'
+ '[' -z /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils ']'
+ local BIGTOP_DETECT_JAVAHOME=
+ for candidate in '"${JSVC_HOME}"' '"${JSVC_HOME}/.."' '"/usr/lib/bigtop-utils"' '"/usr/libexec"'
+ '[' -e /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils/bigtop-detect-javahome ']'
+ BIGTOP_DETECT_JAVAHOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils/bigtop-detect-javahome
+ break
+ '[' -z /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils/bigtop-detect-javahome ']'
+ . /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils/bigtop-detect-javahome
++ BIGTOP_DEFAULTS_DIR=/etc/default
++ '[' -n /etc/default -a -r /etc/default/bigtop-utils ']'
++ JAVA6_HOME_CANDIDATES='\
    /usr/lib/j2sdk1.6-sun \
    /usr/lib/jvm/java-6-sun \
    /usr/lib/jvm/java-1.6.0-sun-1.6.0.* \
    /usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/ \
    /usr/lib/jvm/j2sdk1.6-oracle \
    /usr/lib/jvm/j2sdk1.6-oracle/jre \
    /usr/java/jdk1.6* \
    /usr/java/jre1.6*'
++ OPENJAVA6_HOME_CANDIDATES='\
    /usr/lib/jvm/java-1.6.0-openjdk \
    /usr/lib/jvm/java-1.6.0-openjdk-* \
    /usr/lib/jvm/jre-1.6.0-openjdk*'
++ JAVA7_HOME_CANDIDATES='\
    /usr/java/jdk1.7* \
    /usr/java/jre1.7* \
    /usr/lib/jvm/j2sdk1.7-oracle \
    /usr/lib/jvm/j2sdk1.7-oracle/jre \
    /usr/lib/jvm/java-7-oracle*'
++ OPENJAVA7_HOME_CANDIDATES='\
    /usr/lib/jvm/java-1.7.0-openjdk* \
    /usr/lib/jvm/java-7-openjdk*'
++ JAVA8_HOME_CANDIDATES='\
    /usr/java/jdk1.8* \
    /usr/java/jre1.8* \
    /usr/lib/jvm/j2sdk1.8-oracle \
    /usr/lib/jvm/j2sdk1.8-oracle/jre \
    /usr/lib/jvm/java-8-oracle*'
++ OPENJAVA8_HOME_CANDIDATES='\
    /usr/lib/jvm/java-1.8.0-openjdk* \
    /usr/lib/jvm/java-8-openjdk*'
++ MISCJAVA_HOME_CANDIDATES='\
    /Library/Java/Home \
    /usr/java/default \
    /usr/lib/jvm/default-java \
    /usr/lib/jvm/java-openjdk \
    /usr/lib/jvm/jre-openjdk'
++ case $BIGTOP_JAVA_MAJOR in
++ JAVA_HOME_CANDIDATES='\
    /usr/java/jdk1.7* \
    /usr/java/jre1.7* \
    /usr/lib/jvm/j2sdk1.7-oracle \
    /usr/lib/jvm/j2sdk1.7-oracle/jre \
    /usr/lib/jvm/java-7-oracle*                                \
    /usr/lib/j2sdk1.6-sun \
    /usr/lib/jvm/java-6-sun \
    /usr/lib/jvm/java-1.6.0-sun-1.6.0.* \
    /usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/ \
    /usr/lib/jvm/j2sdk1.6-oracle \
    /usr/lib/jvm/j2sdk1.6-oracle/jre \
    /usr/java/jdk1.6* \
    /usr/java/jre1.6*                                \
    /usr/java/jdk1.8* \
    /usr/java/jre1.8* \
    /usr/lib/jvm/j2sdk1.8-oracle \
    /usr/lib/jvm/j2sdk1.8-oracle/jre \
    /usr/lib/jvm/java-8-oracle*                                \
    /Library/Java/Home \
    /usr/java/default \
    /usr/lib/jvm/default-java \
    /usr/lib/jvm/java-openjdk \
    /usr/lib/jvm/jre-openjdk                             \
    /usr/lib/jvm/java-1.7.0-openjdk* \
    /usr/lib/jvm/java-7-openjdk*                            \
    /usr/lib/jvm/java-1.6.0-openjdk \
    /usr/lib/jvm/java-1.6.0-openjdk-* \
    /usr/lib/jvm/jre-1.6.0-openjdk*                            \
    /usr/lib/jvm/java-1.8.0-openjdk* \
    /usr/lib/jvm/java-8-openjdk*'
++ '[' -z '' ']'
++ for candidate_regex in '$JAVA_HOME_CANDIDATES'
+++ ls -rd '\'
++ for candidate_regex in '$JAVA_HOME_CANDIDATES'
+++ ls -rd /usr/java/jdk1.7.0_67-cloudera
++ for candidate in '`ls -rd $candidate_regex 2>/dev/null`'
++ '[' -e /usr/java/jdk1.7.0_67-cloudera/bin/java ']'
++ export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
++ JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
++ break 2
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_67-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
+ set -x
+ echo 'using 5 as CDH_VERSION'
+ perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/80-sqoop-create-database#g' /var/run/cloudera-scm-agent/process/80-sqoop-create-database/catalina_logging.properties /var/run/cloudera-scm-agent/process/80-sqoop-create-database/cloudera-monitor.properties /var/run/cloudera-scm-agent/process/80-sqoop-create-database/cloudera-stack-monitor.properties /var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties /var/run/cloudera-scm-agent/process/80-sqoop-create-database/sqoop_bootstrap.properties /var/run/cloudera-scm-agent/process/80-sqoop-create-database/sqoop.properties
+ '[' 5 -ge 5 ']'
+ '[' -z /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/meta/cdh_env.sh ']'
+ TOMCAT_CONF=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/../../etc/sqoop2/tomcat-conf.dist
+ export CATALINA_BASE=/var/lib/sqoop2/tomcat-deployment
+ CATALINA_BASE=/var/lib/sqoop2/tomcat-deployment
+ env TOMCAT_CONF=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/../../etc/sqoop2/tomcat-conf.dist TOMCAT_DEPLOYMENT=/var/lib/sqoop2/tomcat-deployment SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2 bash /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/tomcat-deployment.sh
+ export CATALINA_TMPDIR=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/temp
+ CATALINA_TMPDIR=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/temp
+ export CATALINA_OUT=/var/log/sqoop2/sqoop.out
+ CATALINA_OUT=/var/log/sqoop2/sqoop.out
+ export CATALINA_PID=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/sqoop.pid
+ CATALINA_PID=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/sqoop.pid
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh'
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties'
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database'
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true'
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000'
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005'
+ export 'CATALINA_OPTS=-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005 '
+ CATALINA_OPTS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005 '
+ export LOGGING_CONFIG=-Djava.util.logging.config.file=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/catalina_logging.properties
+ LOGGING_CONFIG=-Djava.util.logging.config.file=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/catalina_logging.properties
+ mkdir -p /var/run/cloudera-scm-agent/process/80-sqoop-create-database/temp
+ EXT_LIBS_DIR=/var/lib/sqoop2
+ cp /usr/share/java/mysql-connector-java.jar /var/lib/sqoop2
+ cp /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar /var/lib/sqoop2
+ cp /usr/share/java/oracle-connector-java.jar /var/lib/sqoop2
+ '[' create-database == upgrade -o create-database == create-database ']'
+ export 'JAVA_OPTS=-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005  '
+ JAVA_OPTS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005  '
+ exec /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-tomcat/bin/tool-wrapper.sh -server org.apache.sqoop.tomcat.TomcatToolRunner upgrade


Wed Dec 24 06:04:44 EST 2014
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
using 5 as CDH_VERSION
Sqoop tool executor:
	Version: 1.99.4-cdh5.3.0
	Revision: 75a4ffb64ddc4001a26a04366271b51b262224a1
	Compiled on Tue Dec 16 20:14:23 PST 2014 by jenkins
Running tool: class org.apache.sqoop.tools.tool.UpgradeTool
Upgrade has failed, please check Server logs for further details.
Tool class org.apache.sqoop.tools.tool.UpgradeTool has failed.


19,476 Views
0 Kudos
4
2 ACCEPTED SOLUTIONS

avatar
author-rank hideya
New Contributor

Created ‎01-14-2015 12:06 AM

I've deployed derby.jar and derbyclient.jar contained in db-derby-10.11.1.1-bin.zip from Apache Derby to /var/lib/sqoop2/, and it passed !!

View solution in original post

18,889 Views
4

avatar
author-rank Audi
Contributor

Created ‎02-05-2015 10:46 PM

Hi~Dkumar:

     I have fixed the problem.You can try my step.

1.Download db-derby-10.11.1.1-bin.zip.And install it.

2.Copy derby.jar into /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/jars/

3.Delete /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/webapps/sqoop/WEB-INF/lib/derby-{version}.jar soft link.

4.Make soft link /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/webapps/sqoop/WEB-INF/lib/derby.jar to /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/jars/derby.jar

                                                                           Audi

View solution in original post

16,137 Views
3
19 REPLIES 19

avatar
author-rank Audi
Contributor

Created ‎02-05-2015 05:16 PM

Any one has idea for this issue?

16,749 Views
0 Kudos

avatar
author-rank ossdm
New Contributor

Created ‎02-05-2015 05:46 PM

We overcame the issue, Sqoop2 Service would not start after upgrade, by removing jars (from /var/lib/sqoop2 ) pulled from the Web/latest Apache Derby libs which were a MUST for Sqoop Upgrade then restarting Sqoop Service.

 

 

I ran into issue during upgrade: 5.1.0 to 5.3.0

Latest Apache Derby libs, derbyclient.jar and derby.jar which I pulled from the Web, as per earlier suggestion, took care of the upgrade bug.

But the service would not start. Not until we removed those manually placed for upgrade libraries from /var/lib/sqoop2. Once removed Sqoop service started up just fine. We had case open with Cloudera: 51398 in case you interested.

 

/var/log/sqoop2/derbyrepo.log: start up error

 

Mon Jan 26 13:06:59 PST 2015 Thread[main,5,main] Mon Jan 26 13:06:59 PST 2015 Thread[main,5,main] Startup failed due to missing functionality for org.apache.derby.jdbc.InternalDriver. Please ensure your classpath includes the correct Derby software.

ERROR XBM02: Startup failed due to missing functionality for org.apache.derby.jdbc.InternalDriver. Please ensure your classpath includes the correct Derby software.

                  at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)

                  at org.apache.derby.iapi.services.monitor.Monitor.missingImplementation(Unknown Source)

                  at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)

                  at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)

                  at org.apache.derby.impl.services.monitor.BaseMonitor.startServices(Unknown Source)

                  at org.apache.derby.impl.services.monitor.BaseMonitor.runWithState(Unknown Source)

                  at org.apache.derby.impl.services.monitor.FileMonitor.<init>(Unknown Source)

                  at org.apache.derby.iapi.services.monitor.Monitor.startMonitor(Unknown Source)

                  at org.apache.derby.iapi.jdbc.JDBCBoot.boot(Unknown Source)

                  at org.apache.derby.jdbc.EmbeddedDriver.boot(Unknown Source)

                  at org.apache.derby.jdbc.EmbeddedDriver.<clinit>(Unknown Source)

                  at java.lang.Class.forName0(Native Method)

                  at java.lang.Class.forName(Class.java:190)

                  at org.apache.sqoop.utils.ClassUtils.loadClass(ClassUtils.java:47)

                  at org.apache.sqoop.repository.JdbcRepositoryProvider.initializeRepositoryHandler(JdbcRepositoryProvider.java:122)

                  at org.apache.sqoop.repository.JdbcRepositoryProvider.initialize(JdbcRepositoryProvider.java:65)

                  at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:119)

                  at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:77)

                  at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:53)

                  at org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:41)

                  at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)

                  at org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)

                  at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)

                  at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)

                  at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)

                  at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)

                  at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)

                  at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)

 

var/log/sqoop2: upgrade error

 

2015-01-26 14:52:11,040 ERROR org.apache.sqoop.tools.tool.UpgradeTool: Can't finish upgrading RepositoryManager, Driver and ConnectionManager:

org.apache.sqoop.common.SqoopException: JDBCREPO_0007:Unable to lease link

                  at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:63)

                  at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:85)

                  at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:61)

                  at org.apache.sqoop.repository.JdbcRepository.createOrUpgradeRepository(JdbcRepository.java:127)

                  at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:123)

                  at org.apache.sqoop.tools.tool.UpgradeTool.runToolWithConfiguration(UpgradeTool.java:39)

                  at org.apache.sqoop.tools.ConfiguredTool.runTool(ConfiguredTool.java:35)

                  at org.apache.sqoop.tools.ToolRunner.main(ToolRunner.java:75)

                  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                  at java.lang.reflect.Method.invoke(Method.java:606)

                  at org.apache.sqoop.tomcat.TomcatToolRunner.main(TomcatToolRunner.java:77)

                  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                  at java.lang.reflect.Method.invoke(Method.java:606)

                  at org.apache.catalina.startup.Tool.main(Tool.java:225)

Caused by: java.sql.SQLException: No suitable driver found for jdbc:derby:/var/lib/sqoop2/repository/db;create=true

                  at java.sql.DriverManager.getConnection(DriverManager.java:596)

                  at java.sql.DriverManager.getConnection(DriverManager.java:187)

                  at org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:51)

                  at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:290)

                  at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:840)

                  at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)

                  at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:61)

                  ... 17 more

2015-01-26 14:52:11,043 INFO org.apache.sqoop.core.PropertiesConfigurationProvider: Shutting down configuration poller thread

12,619 Views
0 Kudos

avatar
author-rank abe author-rank
Expert Contributor

Created ‎02-06-2015 05:56 PM

Hey folks,

A few notes on getting Sqoop2 to work and give more detail if there is an error:
1. The Derby jar is required for Sqoop2 to work. It does come packaged with Sqoop2, so do not put it into /var/lib/sqoop2.
2. You can use "set option --name verbose --value true" to see more error information. You may be having different errors for all we know.
12,587 Views
0 Kudos

avatar
Markus Kemper author-rank
Cloudera Employee

Created ‎08-10-2016 02:48 PM

Hello Audi et al,

 

I have seen the error below when the lock files in the Derby database directory are owned by a user other than Sqoop2.  This can happen if you manually connect to the database using the Derby "ij" tool.

 

Server startup failure
org.apache.sqoop.common.SqoopException: JDBCREPO_0007:Unable to lease link
	at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:64)
	<SNIP>
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Could not create a validated object, cause: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.
	at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:1Caused by: java.util.NoSuchElementException: Could not create a validated object, cause: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.

 

# ls -l
total 24
-rw-r--r-- 1 root root 4 Aug 10 11:28 dbex.lck <======
-rw-r--r-- 1 root root 38 Aug 10 11:28 db.lck <=======
drwxr-xr-x 2 sqoop2 sqoop2 4096 Aug 10 10:15 log
drwxr-xr-x 2 sqoop2 sqoop2 4096 Aug 10 10:15 seg0
-rw-r--r-- 1 sqoop2 sqoop2 853 Aug 10 10:15 service.properties
drwxr-xr-x 2 sqoop2 sqoop2 4096 Aug 10 13:49 tmp

 

To get past this issue do the following:

  1. Ensure that there are no connections to the database
  2. Address the lock files
    1. Option 1: chown sqoop2:sqoop2 *.lck
    2. Option 2: rm -rf *.lck
  3. Start Sqoop2

 

Hope this helps,

Markus Kemper - Cloudera Support

9,293 Views
0 Kudos

avatar
author-rank DKumar
Contributor

Created ‎02-05-2015 07:07 PM

Hi Audi,

 

I am still have the same problem. I couldn't find any solution. Still waiting for someone to answer.

12,609 Views
0 Kudos

avatar
author-rank Audi
Contributor

Created ‎02-05-2015 10:30 PM

Hi~Dkumar:

 

12,604 Views
0 Kudos

avatar
author-rank Audi
Contributor

Created ‎02-05-2015 10:46 PM

Hi~Dkumar:

     I have fixed the problem.You can try my step.

1.Download db-derby-10.11.1.1-bin.zip.And install it.

2.Copy derby.jar into /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/jars/

3.Delete /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/webapps/sqoop/WEB-INF/lib/derby-{version}.jar soft link.

4.Make soft link /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/webapps/sqoop/WEB-INF/lib/derby.jar to /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/jars/derby.jar

                                                                           Audi

16,138 Views
3

avatar
author-rank miyazhao
Explorer

Created ‎02-11-2015 01:56 AM

Hi Audi,
Thank you so much ! Great help~~~
12,550 Views
0 Kudos

avatar
author-rank DKay
New Contributor

Created ‎02-19-2015 08:42 PM

Hi,

 

I am facing the exact same issue with Cloudera Manager V. 5.3.1

The folder structure does not exists - /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/jars/ does not exist for my case to copy the derby.jar file.

 

Please guide.

 

Thanks !

DKay

12,498 Views
0 Kudos

avatar
author-rank Audi
Contributor

Created ‎02-24-2015 01:14 AM

 Hi~Dkay:

      Did you install CDH with parcels?If yes, your CDH folder should be in /opt/cloudera/parcels.

Maybe it is /opt/cloudera/parcels/CDH-5.3.1.xxxx/jars.Good luck!

                                                       Audi

12,471 Views
Announcements
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
What's New @ Cloudera
New Action menu item in the Cloudera Operational Database UI
What's New @ Cloudera
Get the list of supported instance types in the Cloudera Ope...
View More Announcements