Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Scalding job failure in CDH5.4

Scalding job failure in CDH5.4

Explorer

 

Hi All,

 

We have the scalding job (v 2.9.2), which was running properly in the CDH 4 cluster. Now, after upgrading to CDH 5.4 we're experiencing the job failures

Is someone familiar with this problem? We suspect that it's related to the jars collisions but moving to the shaded jar didn't resolve the problem

 

Any help will be highly appreciated

Thank you,

Vladi 

 

07:23:54,497 FATAL [IPC Server handler 3 on 55444] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1436182708795_0534_m_000007_1 - exited : java.io.IOException: Split class cascading.tap.hadoop.io.MultiInputSplit not found
	at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:363)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:426)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class cascading.tap.hadoop.io.MultiInputSplit not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2018)
	at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:361)

 

1 REPLY 1

Re: Scalding job failure in CDH5.4

Master Guru
This seems like a job packaging or distributed-cache configuration issue. The scalding/cascading folks may be able to better comment on this, but perhaps the resolutions discussed on https://groups.google.com/d/topic/cascading-user/tUbe_Ikz5I0/discussion may help you?