Reply
Explorer
Posts: 7
Registered: ‎05-04-2015

Scalding job failure in CDH5.4

 

Hi All,

 

We have the scalding job (v 2.9.2), which was running properly in the CDH 4 cluster. Now, after upgrading to CDH 5.4 we're experiencing the job failures

Is someone familiar with this problem? We suspect that it's related to the jars collisions but moving to the shaded jar didn't resolve the problem

 

Any help will be highly appreciated

Thank you,

Vladi 

 

07:23:54,497 FATAL [IPC Server handler 3 on 55444] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1436182708795_0534_m_000007_1 - exited : java.io.IOException: Split class cascading.tap.hadoop.io.MultiInputSplit not found
	at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:363)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:426)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class cascading.tap.hadoop.io.MultiInputSplit not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2018)
	at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:361)

 

Posts: 1,896
Kudos: 433
Solutions: 303
Registered: ‎07-31-2013

Re: Scalding job failure in CDH5.4

This seems like a job packaging or distributed-cache configuration issue. The scalding/cascading folks may be able to better comment on this, but perhaps the resolutions discussed on https://groups.google.com/d/topic/cascading-user/tUbe_Ikz5I0/discussion may help you?
Announcements