Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive JOB failed --map.xml not exist

Hive JOB failed --map.xml not exist

New Contributor

Hi,

We are using Hive over Cassandra on HDP 2.3: For a week, many applications fails with the following exception:

2016-09-15 16:09:05,529 FATAL [AsyncDispatcher event handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.FileNotFoundException: File does not exist: hdfs://cassandraProd/var/opt/hosting/hadoop/hive/scratchdir/pns/248c8712-4eb6-4926-98cf-fdacc54e3425/hive_2016-09-15_16-00-56_218_8691044763703447835-2/-mr-10002/1b934a7e-fdb2-4279-b104-f3970de433a4/map.xml#map.xml
	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:767)
	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createContainerLaunchContext(TaskAttemptImpl.java:812)
	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1533)
	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl$ContainerAssignedTransition.transition(TaskAttemptImpl.java:1510)
	at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
	at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
	at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
	at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:1075)
	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:145)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:1334)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher.handle(MRAppMaster.java:1326)
	at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:175)
	at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:108)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File does not exist: hdfs://cassandraProd/var/opt/hosting/hadoop/hive/scratchdir/pns/248c8712-4eb6-4926-98cf-fdacc54e3425/hive_2016-09-15_16-00-56_218_8691044763703447835-2/-mr-10002/1b934a7e-fdb2-4279-b104-f3970de433a4/map.xml#map.xml
	at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1309)
	at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301)
	at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:751)
	at org.apache.hadoop.mapreduce.v2.util.MRApps.parseDistributedCacheArtifacts(MRApps.java:571)
	at org.apache.hadoop.mapreduce.v2.util.MRApps.setupDistributedCache(MRApps.java:463)
	at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.createCommonContainerLaunchContext(TaskAttemptImpl.java:693)
	... 14 more
2016-09-15 16:09:05,532 INFO [AsyncDispatcher ShutDown handler] org.apache.hadoop.yarn.event.AsyncDispatcher: Exiting, bbye..

I have checked hive server logs and it also shows same error message as above.

Any kind of help would be greatly appreciated.

Best regards,

2 REPLIES 2

Re: Hive JOB failed --map.xml not exist

Contributor

as in your logs it looks like following file does not exist on hdfs

hdfs://cassandraProd/var/opt/hosting/hadoop/hive/scratchdir/pns/248c8712-4eb6-4926-98cf-fdacc54e3425/hive_2016-09-15_16-00-56_218_8691044763703447835-2/-mr-10002/1b934a7e-fdb2-4279-b104-f3970de433a4/map.xml#map.xml

to check if file exist or not do

#hadoop fs -ls /cassandraProd/var/opt/hosting/hadoop/hive/scratchdir/pns/248c8712-4eb6-4926-98cf-fdacc54e3425/hive_2016-09-15_16-00-56_218_8691044763703447835-2/-mr-10002/1b934a7e-fdb2-4279-b104-f3970de433a4/

Re: Hive JOB failed --map.xml not exist

New Contributor

I checked, the file not exists on hdfs!!