Support Questions
Find answers, ask questions, and share your expertise

Sqoop import: org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler

Highlighted

Sqoop import: org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler

One of our Sqoop import jobs throwing below error, Please help me to fix this issu.

2017-12-11 06:40:15,203 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1507175565630_261578_m_000000_0 TaskAttempt Transitioned from RUNNING to SUCCESS_CONTAINER_CLEANUP
2017-12-11 06:40:15,203 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_CLEANUP for container container_e106_1507175565630_261578_01_000002 taskAttempt attempt_1507175565630_261578_m_000000_0
2017-12-11 06:40:15,204 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1507175565630_261578_m_000000_0
2017-12-11 06:40:15,204 INFO [ContainerLauncher #1] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: Opening proxy : clpd863:45454
2017-12-11 06:40:15,216 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1507175565630_261578_m_000000_0 TaskAttempt Transitioned from SUCCESS_CONTAINER_CLEANUP to SUCCEEDED
2017-12-11 06:40:15,220 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1507175565630_261578_m_000000_0
2017-12-11 06:40:15,221 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1507175565630_261578_m_000000 Task Transitioned from RUNNING to SUCCEEDED
2017-12-11 06:40:15,222 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1
2017-12-11 06:40:15,223 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1507175565630_261578Job Transitioned from RUNNING to COMMITTING
2017-12-11 06:40:15,223 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_COMMIT
2017-12-11 06:40:15,235 ERROR [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Could not commit job
java.io.FileNotFoundException: File hdfs://Intra1cluster/intraday/stage/lte/vertica/rops/fdd5/_temporary/1 does not exist.
	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:904)
	at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:113)
	at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:966)
	at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:962)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:962)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1534)
	at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1574)
	at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getAllCommittedTaskPaths(FileOutputCommitter.java:321)
	at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitJobInternal(FileOutputCommitter.java:391)
	at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitJob(FileOutputCommitter.java:364)
	at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:285)
	at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:237)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745) 
1 REPLY 1
Highlighted

Re: Sqoop import: org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler

Contributor

@sivareddy akkala can you post the sqoop job command here and full logs. The error states its unable to find the file in hdfs location -

java.io.FileNotFoundException: File hdfs://Intra1cluster/intraday/stage/lte/vertica/rops/fdd5/_temporary/1 does not exist.