Testing CM Installation: PiEstimator pending/unassigned for hours

[#1]

I was following the Installation Test guide below:

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Installat...

I ran this command from one of my host:

sudo -u hdfs hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 100

This is what I get:

[apps@analyticpapp2 ~]$ sudo -u hdfs hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 100
Number of Maps = 10
Samples per Map = 100
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
14/09/09 10:09:53 INFO client.RMProxy: Connecting to ResourceManager at analyticpapp1/xx.x.xxx.xx:8032
14/09/09 10:09:54 INFO input.FileInputFormat: Total input paths to process : 10
14/09/09 10:09:54 INFO mapreduce.JobSubmitter: number of splits:10
14/09/09 10:09:55 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1410232070158_0001
14/09/09 10:09:55 INFO impl.YarnClientImpl: Submitted application application_1410232070158_0001
14/09/09 10:09:55 INFO mapreduce.Job: The url to track the job: http://analyticpapp1:8088/proxy/application_1410232070158_0001/
14/09/09 10:09:55 INFO mapreduce.Job: Running job: job_1410232070158_0001

and it just stops. nothing happen.

My cluster consist of 10 hosts.

On 9 hosts, I've set 6 services on each:

- HBase regionserver

- HDFS datanode

- Hive gateway

- Impala daemon

- Spark worker

- Yarn nodemanager

On 1 Host (Head Node), I run the following services:

- HBase master

- HDFS namenode

- HDFS secondarynamenode

- Hive hivemetastore

- Hive hiveserver2

- Hive gateway

- Hue server

- Impala catalogserver

- Impala statestore

- HBase indexer

- Oozie server

- Solr server

- Spark master

- Sqoop server

- Yarn jobhistory

- Yarn resourcemanager

- Zookeeper server

Any advice?

Thanks in advanced.

[#2 · Accepted solution]

Solved this issue.

My problem was simple, My NodeManager log directory setting is not pointing to the container.

yarn.nodemanager.log-dirs=/apps/ext/var/log/hadoop-yarn

it was suppopsed to be

yarn.nodemanager.log-dirs=/apps/ext/var/log/hadoop-yarn/container

I figured it out after I see this message on my NodeManager web ui, under Node Log information, "NodeHealthReport 1/1 log-dirs turned bad"

I was able to run and finish the PiEstimator after I update the log-dirs and restart the service.