Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Yarn memory is 100% when loading data into hadoop cluster from Mysql through Sqoop and cant access yarn config in Ambari UI.

Yarn memory is 100% when loading data into hadoop cluster from Mysql through Sqoop and cant access yarn config in Ambari UI.

New Contributor

FYI:- My cluster config Nodes = 5 (1 Master, 4 DN ) memory = 64 GB on each node Processors = 64 on each node 1 TB on each node (1 Disk * 5 TB) Yarn memory 250GB.

7 REPLIES 7
Highlighted

Re: Yarn memory is 100% when loading data into hadoop cluster from Mysql through Sqoop and cant access yarn config in Ambari UI.

Contributor

how much is your source data size ? based on that mappers will be spun. if you need to control the number of mappers - please set the property to

-m,--num-mappers <n> Use n map tasks to import in parallel

Re: Yarn memory is 100% when loading data into hadoop cluster from Mysql through Sqoop and cant access yarn config in Ambari UI.

New Contributor

My table has around 5 crore records and contains 12 columns. Iam using --num-mappers 20. still facing a issue like as below.

16/12/14 09:41:52 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/medwadmin/.staging/job_1481181930400_0211 16/12/14 09:41:53 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: java.sql.SQLException: Incorrect key file for table '/tmp/#sql_66aa_0.MYI'; try to repair it at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:178) at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301) at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308) at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:196) at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:169) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:266) at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:729) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:499) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) at org.apache.sqoop.Sqoop.run(Sqoop.java:148) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235) at org.apache.sqoop.Sqoop.main(Sqoop.java:244) Caused by: java.sql.SQLException: Incorrect key file for table '/tmp/#sql_66aa_0.MYI'; try to repair it at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4187) at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2143) at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1999) at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3493) at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:490) at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3198) at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2366) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2789) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2814) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2763) at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1614) at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:149) ... 22 more

Re: Yarn memory is 100% when loading data into hadoop cluster from Mysql through Sqoop and cant access yarn config in Ambari UI.

Contributor

@kotesh banoth This is error thrown from mysql db. Check it in the MYSQL server box, surely there seems to be a space or permission issue

: java.sql.SQLException: Incorrect key file for table '/tmp/#sql_66aa_0.MYI'; try to repair it at

Re: Yarn memory is 100% when loading data into hadoop cluster from Mysql through Sqoop and cant access yarn config in Ambari UI.

Contributor

1) YARN memory at 100% is a good thing. That implies you are fully utilizing your cluster.

2) Can you show a screenshot of your Ambari UI where you cannot access the YARN -> Config tab?

3) "Incorrect key file for table '/tmp/" appears to be an issue with your MySQL table, not anything to do with your Hadoop config or Sqoop: http://stackoverflow.com/questions/2090073/mysql-incorrect-key-file-for-tmp-table-when-making-multip... . Possibly due to lack of /tmp space. Please correct the MySQL issue and retest.

4) You can typically see a copy of the SQL which Sqoop is attempting to run as part of its normal output when you run the command. Try running this command directly in MySQL to confirm whether or not this is a MySQL issue.

Re: Yarn memory is 100% when loading data into hadoop cluster from Mysql through Sqoop and cant access yarn config in Ambari UI.

New Contributor

i had posted screen shots above please check

Re: Yarn memory is 100% when loading data into hadoop cluster from Mysql through Sqoop and cant access yarn config in Ambari UI.

New Contributor

@Wes Floyd please find the screen shots10365-yarn.png

The above pic where i cant access yarn configs.

The below one is cluster image with YARN memory full.

10366-cluster.png

Re: Yarn memory is 100% when loading data into hadoop cluster from Mysql through Sqoop and cant access yarn config in Ambari UI.

Contributor

Do you have NodeManager running on all machines, including the "Master" server running Ambari? This would explain why you are unable to use Ambari correctly while running YARN jobs (Sqoop).

Go to the Hosts tab, search for the NodeManager component across your hosts. Find out which hosts are running the NodeManager service.