Reply
Highlighted
Champion
Posts: 562
Registered: ‎05-16-2016

Re: jpb submitted to mapreduce in Yarn is stuck while ingesting data using SQOOP

[ Edited ]

Why is you cluster has  "red" I am suspecting some disk space I am only guessing just run the host health check also 

whats the parameter you had put in 

yarn.nodemanager.resource.memory-mb
yarn.scheduler.minimum-allocation-mb<
mapreduce.map.memory.mb
mapreduce.reduce.memory.mb

 

Explorer
Posts: 17
Registered: ‎06-03-2017

Re: jpb submitted to mapreduce in Yarn is stuck while ingesting data using SQOOP

 

 

 <property>
    <name>yarn.scheduler.minimum-allocation-mb</name>
    <value>1024</value>
  </property>

 

 

in /hadoop/ conf.cloudera.yarn/map-redsite.xml it the following are set to 0, so I changed them to what is in /haddop/conf/map-redsite.xm

 

  <property>
    <name>mapreduce.map.memory.mb</name>
    <value>1024</value>
  </property>
  <property>
    <name>mapreduce.map.cpu.vcores</name>
    <value>1</value>
  </property>
  <property>
    <name>mapreduce.reduce.memory.mb</name>
    <value>1024</value>
  </property>

I could not find the parameter

yarn.nodemanager.resource.memory-mb

 

It is still at same point, no progress.

 

 

 

 

Explorer
Posts: 17
Registered: ‎06-03-2017

Re: jpb submitted to mapreduce in Yarn is stuck while ingesting data using SQOOP

Champion
Posts: 562
Registered: ‎05-16-2016

Re: jpb submitted to mapreduce in Yarn is stuck while ingesting data using SQOOP

As pointed mostly like it will be the resource allocation on those parameter good that you found it in that thread .I was close to narrow it down . Guess what sometimes it could be some socket configuration on the OS too . The logs will clearly guide us to it .  

Announcements