Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Creating Sqoop 2 Database

Solved Go to solution

Creating Sqoop 2 Database

Explorer

hi all,

 

when i try to install sqoop2, its giving an error.

 

Failed to execute command Create Sqoop Database on service Sqoop 2

 

Wed Dec 24 06:04:44 EST 2014
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/meta/cdh_env.sh ']'
+ OLD_IFS=' 	
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS=' 	
'
+ COUNT=1
++ seq 1 1
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.3.0-1.cdh5.3.0.p0.30
+ . /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.3.0-1.cdh5.3.0.p0.30
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-kms
+ locate_cdh_java_home
+ '[' -z '' ']'
+ '[' -z /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils ']'
+ local BIGTOP_DETECT_JAVAHOME=
+ for candidate in '"${JSVC_HOME}"' '"${JSVC_HOME}/.."' '"/usr/lib/bigtop-utils"' '"/usr/libexec"'
+ '[' -e /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils/bigtop-detect-javahome ']'
+ BIGTOP_DETECT_JAVAHOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils/bigtop-detect-javahome
+ break
+ '[' -z /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils/bigtop-detect-javahome ']'
+ . /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-utils/bigtop-detect-javahome
++ BIGTOP_DEFAULTS_DIR=/etc/default
++ '[' -n /etc/default -a -r /etc/default/bigtop-utils ']'
++ JAVA6_HOME_CANDIDATES='\
    /usr/lib/j2sdk1.6-sun \
    /usr/lib/jvm/java-6-sun \
    /usr/lib/jvm/java-1.6.0-sun-1.6.0.* \
    /usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/ \
    /usr/lib/jvm/j2sdk1.6-oracle \
    /usr/lib/jvm/j2sdk1.6-oracle/jre \
    /usr/java/jdk1.6* \
    /usr/java/jre1.6*'
++ OPENJAVA6_HOME_CANDIDATES='\
    /usr/lib/jvm/java-1.6.0-openjdk \
    /usr/lib/jvm/java-1.6.0-openjdk-* \
    /usr/lib/jvm/jre-1.6.0-openjdk*'
++ JAVA7_HOME_CANDIDATES='\
    /usr/java/jdk1.7* \
    /usr/java/jre1.7* \
    /usr/lib/jvm/j2sdk1.7-oracle \
    /usr/lib/jvm/j2sdk1.7-oracle/jre \
    /usr/lib/jvm/java-7-oracle*'
++ OPENJAVA7_HOME_CANDIDATES='\
    /usr/lib/jvm/java-1.7.0-openjdk* \
    /usr/lib/jvm/java-7-openjdk*'
++ JAVA8_HOME_CANDIDATES='\
    /usr/java/jdk1.8* \
    /usr/java/jre1.8* \
    /usr/lib/jvm/j2sdk1.8-oracle \
    /usr/lib/jvm/j2sdk1.8-oracle/jre \
    /usr/lib/jvm/java-8-oracle*'
++ OPENJAVA8_HOME_CANDIDATES='\
    /usr/lib/jvm/java-1.8.0-openjdk* \
    /usr/lib/jvm/java-8-openjdk*'
++ MISCJAVA_HOME_CANDIDATES='\
    /Library/Java/Home \
    /usr/java/default \
    /usr/lib/jvm/default-java \
    /usr/lib/jvm/java-openjdk \
    /usr/lib/jvm/jre-openjdk'
++ case $BIGTOP_JAVA_MAJOR in
++ JAVA_HOME_CANDIDATES='\
    /usr/java/jdk1.7* \
    /usr/java/jre1.7* \
    /usr/lib/jvm/j2sdk1.7-oracle \
    /usr/lib/jvm/j2sdk1.7-oracle/jre \
    /usr/lib/jvm/java-7-oracle*                                \
    /usr/lib/j2sdk1.6-sun \
    /usr/lib/jvm/java-6-sun \
    /usr/lib/jvm/java-1.6.0-sun-1.6.0.* \
    /usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/ \
    /usr/lib/jvm/j2sdk1.6-oracle \
    /usr/lib/jvm/j2sdk1.6-oracle/jre \
    /usr/java/jdk1.6* \
    /usr/java/jre1.6*                                \
    /usr/java/jdk1.8* \
    /usr/java/jre1.8* \
    /usr/lib/jvm/j2sdk1.8-oracle \
    /usr/lib/jvm/j2sdk1.8-oracle/jre \
    /usr/lib/jvm/java-8-oracle*                                \
    /Library/Java/Home \
    /usr/java/default \
    /usr/lib/jvm/default-java \
    /usr/lib/jvm/java-openjdk \
    /usr/lib/jvm/jre-openjdk                             \
    /usr/lib/jvm/java-1.7.0-openjdk* \
    /usr/lib/jvm/java-7-openjdk*                            \
    /usr/lib/jvm/java-1.6.0-openjdk \
    /usr/lib/jvm/java-1.6.0-openjdk-* \
    /usr/lib/jvm/jre-1.6.0-openjdk*                            \
    /usr/lib/jvm/java-1.8.0-openjdk* \
    /usr/lib/jvm/java-8-openjdk*'
++ '[' -z '' ']'
++ for candidate_regex in '$JAVA_HOME_CANDIDATES'
+++ ls -rd '\'
++ for candidate_regex in '$JAVA_HOME_CANDIDATES'
+++ ls -rd /usr/java/jdk1.7.0_67-cloudera
++ for candidate in '`ls -rd $candidate_regex 2>/dev/null`'
++ '[' -e /usr/java/jdk1.7.0_67-cloudera/bin/java ']'
++ export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
++ JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
++ break 2
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_67-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
+ set -x
+ echo 'using 5 as CDH_VERSION'
+ perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/80-sqoop-create-database#g' /var/run/cloudera-scm-agent/process/80-sqoop-create-database/catalina_logging.properties /var/run/cloudera-scm-agent/process/80-sqoop-create-database/cloudera-monitor.properties /var/run/cloudera-scm-agent/process/80-sqoop-create-database/cloudera-stack-monitor.properties /var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties /var/run/cloudera-scm-agent/process/80-sqoop-create-database/sqoop_bootstrap.properties /var/run/cloudera-scm-agent/process/80-sqoop-create-database/sqoop.properties
+ '[' 5 -ge 5 ']'
+ '[' -z /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/meta/cdh_env.sh ']'
+ TOMCAT_CONF=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/../../etc/sqoop2/tomcat-conf.dist
+ export CATALINA_BASE=/var/lib/sqoop2/tomcat-deployment
+ CATALINA_BASE=/var/lib/sqoop2/tomcat-deployment
+ env TOMCAT_CONF=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/../../etc/sqoop2/tomcat-conf.dist TOMCAT_DEPLOYMENT=/var/lib/sqoop2/tomcat-deployment SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2 bash /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/tomcat-deployment.sh
+ export CATALINA_TMPDIR=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/temp
+ CATALINA_TMPDIR=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/temp
+ export CATALINA_OUT=/var/log/sqoop2/sqoop.out
+ CATALINA_OUT=/var/log/sqoop2/sqoop.out
+ export CATALINA_PID=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/sqoop.pid
+ CATALINA_PID=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/sqoop.pid
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh'
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties'
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database'
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true'
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000'
+ OPTIONS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005'
+ export 'CATALINA_OPTS=-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005 '
+ CATALINA_OPTS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005 '
+ export LOGGING_CONFIG=-Djava.util.logging.config.file=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/catalina_logging.properties
+ LOGGING_CONFIG=-Djava.util.logging.config.file=/var/run/cloudera-scm-agent/process/80-sqoop-create-database/catalina_logging.properties
+ mkdir -p /var/run/cloudera-scm-agent/process/80-sqoop-create-database/temp
+ EXT_LIBS_DIR=/var/lib/sqoop2
+ cp /usr/share/java/mysql-connector-java.jar /var/lib/sqoop2
+ cp /usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar /var/lib/sqoop2
+ cp /usr/share/java/oracle-connector-java.jar /var/lib/sqoop2
+ '[' create-database == upgrade -o create-database == create-database ']'
+ export 'JAVA_OPTS=-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005  '
+ JAVA_OPTS='-Xms163577856 -Xmx163577856  -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dlog4j.configuration=file:////var/run/cloudera-scm-agent/process/80-sqoop-create-database/log4j.properties -Dsqoop.config.dir=/var/run/cloudera-scm-agent/process/80-sqoop-create-database -Djava.net.preferIPv4Stack=true -Dsqoop.http.port=12000 -Dsqoop.admin.port=8005  '
+ exec /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/bigtop-tomcat/bin/tool-wrapper.sh -server org.apache.sqoop.tomcat.TomcatToolRunner upgrade


Wed Dec 24 06:04:44 EST 2014
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
using 5 as CDH_VERSION
Sqoop tool executor:
	Version: 1.99.4-cdh5.3.0
	Revision: 75a4ffb64ddc4001a26a04366271b51b262224a1
	Compiled on Tue Dec 16 20:14:23 PST 2014 by jenkins
Running tool: class org.apache.sqoop.tools.tool.UpgradeTool
Upgrade has failed, please check Server logs for further details.
Tool class org.apache.sqoop.tools.tool.UpgradeTool has failed.


2 ACCEPTED SOLUTIONS

Accepted Solutions

Re: Creating Sqoop 2 Database

New Contributor

I've deployed derby.jar and derbyclient.jar contained in db-derby-10.11.1.1-bin.zip from Apache Derby to /var/lib/sqoop2/, and it passed !!

Re: Creating Sqoop 2 Database

Explorer

Hi~Dkumar:

     I have fixed the problem.You can try my step.

1.Download db-derby-10.11.1.1-bin.zip.And install it.

2.Copy derby.jar into /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/jars/

3.Delete /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/webapps/sqoop/WEB-INF/lib/derby-{version}.jar soft link.

4.Make soft link /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2/webapps/sqoop/WEB-INF/lib/derby.jar to /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/jars/derby.jar

                                                                           Audi

19 REPLIES 19

Re: Creating Sqoop 2 Database

What do the role logs say?

It sounds like this is a brand new Sqoop 2 (not an upgrade). Are you sure you gave it an empty database? Check /var/lib/sqoop2 (or whatever you've configured for "Sqoop 2 Server Metastore Directory") for any leftover data that may be causing a problem.

Thanks,
Darren

Re: Creating Sqoop 2 Database

New Contributor

I'm having the same problem while deploying services within my cluster.

when it comes to the creation of the DB, it fails.

Capture_sqoop2_error.JPG

here is the content of the config log:

2015-01-02 11:48:18,428 INFO org.apache.sqoop.core.PropertiesConfigurationProvider: Shutting down configuration poller thread
2015-01-02 11:54:26,291 INFO org.apache.sqoop.core.PropertiesConfigurationProvider: Starting config file poller thread
2015-01-02 11:54:26,293 INFO org.apache.sqoop.tools.tool.UpgradeTool: Initializing the RepositoryManager with immutable option turned off.
2015-01-02 11:54:26,296 INFO org.apache.sqoop.repository.RepositoryManager: Setting system properties: {derby.stream.error.file=/var/log/sqoop2/derbyrepo.log}
2015-01-02 11:54:26,307 INFO org.apache.sqoop.repository.JdbcRepositoryContext: [repo-ctx] handler=org.apache.sqoop.repository.derby.DerbyRepositoryHandler, conn-url=jdbc:derby:/var/lib/sqoop2/repository/db;create=true, driver=org.apache.derby.jdbc.EmbeddedDriver, user=null, password=*****, jdbc-props={}, tx-isolation=READ_COMMITTED, max-conn=10
2015-01-02 11:54:26,707 INFO org.apache.sqoop.repository.derby.DerbyRepositoryHandler: DerbyRepositoryHandler initialized.
2015-01-02 11:54:26,711 INFO org.apache.sqoop.repository.JdbcRepositoryProvider: JdbcRepositoryProvider initialized
2015-01-02 11:54:26,712 INFO org.apache.sqoop.repository.JdbcRepositoryProvider: JdbcRepository initialized.
2015-01-02 11:54:26,712 INFO org.apache.sqoop.repository.RepositoryManager: Creating or updating respository at bootup
2015-01-02 11:54:26,718 ERROR org.apache.sqoop.tools.tool.UpgradeTool: Can't finish upgrading RepositoryManager, Driver and ConnectionManager:
org.apache.sqoop.common.SqoopException: JDBCREPO_0007:Unable to lease link
	at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:63)
	at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:85)
	at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:61)
	at org.apache.sqoop.repository.JdbcRepository.createOrUpgradeRepository(JdbcRepository.java:127)
	at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:123)
	at org.apache.sqoop.tools.tool.UpgradeTool.runToolWithConfiguration(UpgradeTool.java:39)
	at org.apache.sqoop.tools.ConfiguredTool.runTool(ConfiguredTool.java:35)
	at org.apache.sqoop.tools.ToolRunner.main(ToolRunner.java:75)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.sqoop.tomcat.TomcatToolRunner.main(TomcatToolRunner.java:77)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.catalina.startup.Tool.main(Tool.java:225)
Caused by: java.sql.SQLException: No suitable driver found for jdbc:derby:/var/lib/sqoop2/repository/db;create=true
	at java.sql.DriverManager.getConnection(DriverManager.java:596)
	at java.sql.DriverManager.getConnection(DriverManager.java:187)
	at org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:51)
	at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:290)
	at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:840)
	at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
	at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:61)
	... 17 more
2015-01-02 11:54:26,722 INFO org.apache.sqoop.core.PropertiesConfigurationProvider: Shutting down configuration poller thread

 

I checked different directories and can't find any problem. I can even see the file "postgresql-9.0-801.jdbc4.jar" in /var/lib/sqoop2.

Any solution is welcomed!!!

Regards

 

Re: Creating Sqoop 2 Database

New Contributor

Any help????

Re: Creating Sqoop 2 Database

New Contributor

I have the same problem.

 

It probably stops because of

  "No suitable driver found for jdbc:derby:/var/lib/sqoop2/repository/db;create=tr​ue"

 

Does anyone know what library I should put it to where ?

 

best regards!

 

Re: Creating Sqoop 2 Database

New Contributor

I've deployed derby.jar and derbyclient.jar contained in db-derby-10.11.1.1-bin.zip from Apache Derby to /var/lib/sqoop2/, and it passed !!

Re: Creating Sqoop 2 Database

Explorer

Hi All,

 

The Create database worked succesfully. But, when I try to start the Sqoop2, its giving errors. Can someone help me on this ?

 

1)    Service did not start successfully; not all of the required roles started: Service has only 0 Sqoop 2 Server roles running instead of minimum required 1.

 

 

2) 

 

Server startup failure
org.apache.sqoop.common.SqoopException: JDBCREPO_0007:Unable to lease link
	at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:63)
	at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:85)
	at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:61)
	at org.apache.sqoop.repository.JdbcRepository.isRespositorySuitableForUse(JdbcRepository.java:142)
	at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:128)
	at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:77)
	at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:53)
	at org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:41)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)
	at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)
	at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)
	at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
	at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
	at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
	at org.apache.catalina.core.StandardService.start(StandardService.java:525)
	at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: java.sql.SQLException: No suitable driver found for jdbc:derby:/var/lib/sqoop2/repository/db;create=true
	at java.sql.DriverManager.getConnection(DriverManager.java:596)
	at java.sql.DriverManager.getConnection(DriverManager.java:187)
	at org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:78)
	at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
	at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148)
	at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
	at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:61)
	... 31 more
Sqoop server failed to start
java.lang.RuntimeException: Failure in server initialization
	at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:60)
	at org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:41)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)
	at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)
	at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)
	at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
	at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
	at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
	at org.apache.catalina.core.StandardService.start(StandardService.java:525)
	at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: org.apache.sqoop.common.SqoopException: JDBCREPO_0007:Unable to lease link
	at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:63)
	at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:85)
	at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:61)
	at org.apache.sqoop.repository.JdbcRepository.isRespositorySuitableForUse(JdbcRepository.java:142)
	at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:128)
	at org.apache.sqoop.repository.RepositoryManager.initialize(RepositoryManager.java:77)
	at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:53)
	... 25 more
Caused by: java.sql.SQLException: No suitable driver found for jdbc:derby:/var/lib/sqoop2/repository/db;create=true
	at java.sql.DriverManager.getConnection(DriverManager.java:596)
	at java.sql.DriverManager.getConnection(DriverManager.java:187)
	at org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:78)
	at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
	at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1148)
	at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
	at org.apache.sqoop.repository.JdbcRepositoryTransaction.begin(JdbcRepositoryTransaction.java:61)
	... 31 more

Re: Creating Sqoop 2 Database

Explorer

[root@hmaster local]# sqoop2
Sqoop home directory: /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/sqoop2
Jan 26, 2015 8:18:33 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Sqoop Shell: Type 'help' or '\h' for help.

sqoop:000> show version --all
client version:
Sqoop 1.99.4-cdh5.3.0 source revision 75a4ffb64ddc4001a26a04366271b51b262224a1
Compiled by jenkins on Tue Dec 16 20:14:23 PST 2014
Exception has occurred during processing command
Exception: org.apache.sqoop.common.SqoopException Message: CLIENT_0000:An unknown error has occurred
sqoop:000>

Highlighted

Re: Creating Sqoop 2 Database

New Contributor

Thank you very much for the solution. It helped me to resolve same issue.

Re: Creating Sqoop 2 Database

Explorer

Hi Dkumar:

  I have the same problem.My sqoop 2 service did not start successfully;and it returns the same message like yours.

Did you fix the problem?Can you give me some idea for the problem?Thank you.

Don't have an account?
Coming from Hortonworks? Activate your account here