Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

sqoop import - Cannot initialize cluster

sqoop import - Cannot initialize cluster

New Contributor

I am new to Hadoop and Sqoop and am trying to import mySql table to Hadoop.

My environment:

Ubuntu 16.04

Hadoop 2.6.0-cdh5.16.1

Sqoop 1.4.7

 

While trying to import the mySql table, I get an error

 

$ sqoop import --connect jdbc:mysql://127.0.0.1/crawl_data_stats --username root --password password --table auth_group

...
ERROR tool.ImportTool: Import failed: java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
	at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
	at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
	at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
	at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1260)
	at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1256)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
	at org.apache.hadoop.mapreduce.Job.connect(Job.java:1256)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1284)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
	at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:200)
	at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:173)
	at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:270)
	at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
	at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
10 REPLIES 10

Re: sqoop import - Cannot initialize cluster

Expert Contributor

Hello @NadeemMehraj,

 

As I am not sure about the components installed in your environment. You may try to set classic as a value for mapreduce.framework.name parameter in mapred-site.xml.

 

Hope that helps.

Re: sqoop import - Cannot initialize cluster

New Contributor

Thanks @Consult 

I changed mapred-site.xml file.

However the error persists

Re: sqoop import - Cannot initialize cluster

New Contributor

Thanks @Consult 
It seems YARN is installed.

$ yarn version





*Hadoop 2.6.0-cdh5.16.1Subversion http://github.com/cloudera/hadoop
-r
716da186d55cd13037ae0e309c904218b03ed19d Compiled by jenkins on
2018-11-22T06:46ZCompiled with protoc 2.5.0From source with checksum
14b87e817bcbb3895b4c0ae861a99eThis command was run using
/usr/lib/hadoop/hadoop-common-2.6.0-cdh5.16.1.jar*

Highlighted

Re: sqoop import - Cannot initialize cluster

Expert Contributor

Hello @NadeemMehraj,

 

I am assuming you have access to CM & with your installed yarn, change mapreduce.framework.name to yarn. For reference on exact syntax to update that value, you can refer Configure Properties for YARN Clusters.

 

And restart the services that CM suggests after updating the config.

Re: sqoop import - Cannot initialize cluster

New Contributor
Thanks
I went through steps mentioned in
https://www.cloudera.com/documentation/enterprise/5-4-x/topics/cdh_ig_yarn_cluster_deploy.html#topic...

Upon reaching Step 8: Create the history Directory and Set Permissions and
Owner
I get the following output
$ sudo -u hdfs hadoop fs -mkdir -p /user/history
[sudo] password for applied:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
Warning: fs.defaultFS is not set when running "mkdir" command.
mkdir: `/user/history': Input/output error

Re: sqoop import - Cannot initialize cluster

Expert Contributor

Hello @NadeemMehraj,

 

Is there a particular reason you are following those steps? If not then pertaining to this subject, I would just to start by testing by changing just below value of mapreduce.framework.name parameters. Following is the update needed on your mapred-site.xml.

 

<property>
 <name>mapreduce.framework.name</name>
 <value>yarn</value>
</property>

 

Re: sqoop import - Cannot initialize cluster

New Contributor

Oh I thought that I need to follow steps in that link. I will remove all changes except that in step 1 and check

Re: sqoop import - Cannot initialize cluster

New Contributor

I removed all the configuration except that mentioned in step 1 for mapreduce.framework.name.

However I still get the same error

Re: sqoop import - Cannot initialize cluster

New Contributor

I removed all the configuration changes except the one mentioned in step 1 for mapreduce.framework.name

However the error still persists

Don't have an account?
Coming from Hortonworks? Activate your account here