Support Questions
Find answers, ask questions, and share your expertise

HBase MapReduce Job - Error: java.io.IOException: Unable to initialize any output collector

HBase MapReduce Job - Error: java.io.IOException: Unable to initialize any output collector

Champion Alumni

Hello,

 

I'm trying to do a map reduce job that writes into a hbase table. However,

- the reducer is never called.

- the action is considered as successful in oozie workflow

 

The complete error (is showed into the logs, but not considered as an error within oozie):

Error: java.io.IOException: Unable to initialize any output collector
  at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:412)
  at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
  at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695)
  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

 

Note: not only that the reduce code is not called, but the HBase table is not created.

 

I saw something else that had the same error, but nobody answered..http://stackoverflow.com/questions/26926571/cdh5-2-mr-unable-to-initialize-any-output-collector 

 

Where should I look for? Do you need any other details about my map reduce job? 

 

Thank you!

 

GHERMAN Alina
2 REPLIES 2

Re: HBase MapReduce Job - Error: java.io.IOException: Unable to initialize any output collector

Master Guru
The following is what I've answered on your StackOverflow post, hope it helped:

Checking the full task log of the failed attempt attempt_1415950730849_0001_m_000000_0 will help tell why you ran into the given exception.

The most common reason of observing such an error is a misconfigured value of io.sort.mb in your job. Its value must never be anywhere close to (or higher than) the configured map task heap size, and must also not currently exceed ~2000 MB (Java array maximum size).

An upstream improvement of making the error more clear on the true failure was also filed and resolved recently, via MAPREDUCE-6194.

Re: HBase MapReduce Job - Error: java.io.IOException: Unable to initialize any output collector

Champion Alumni

I just had this error once more time. This time it was because I added a compression codec that wasn't installed on my system.

 

GHERMAN Alina