Reply
Expert Contributor
Posts: 109
Registered: ‎05-19-2016

Re: Replica Not FoundException

[ Edited ]

Thank you. Actually my sqoop jobs are randomly getting killed with this message: @saranvisa 

 

Do you have some idea why could this be happening?

 

2018-05-21 14:55:27,984 [main] INFO  org.apache.hadoop.mapreduce.Job  - Counters: 30
	File System Counters
		FILE: Number of bytes read=0
		FILE: Number of bytes written=3184370
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=1201
		HDFS: Number of bytes written=60820
		HDFS: Number of read operations=40
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=20
	Job Counters 
		Launched map tasks=10
		Other local map tasks=10
		Total time spent by all maps in occupied slots (ms)=139947
		Total time spent by all reduces in occupied slots (ms)=0
		Total time spent by all map tasks (ms)=139947
		Total vcore-seconds taken by all map tasks=139947
		Total megabyte-seconds taken by all map tasks=143305728
	Map-Reduce Framework
		Map input records=206
		Map output records=206
		Input split bytes=1201
		Spilled Records=0
		Failed Shuffles=0
		Merged Map outputs=0
		GC time elapsed (ms)=930
		CPU time spent (ms)=17940
		Physical memory (bytes) snapshot=3550318592
		Virtual memory (bytes) snapshot=16373563392
		Total committed heap usage (bytes)=8241807360
	File Input Format Counters 
		Bytes Read=0
	File Output Format Counters 
		Bytes Written=60820
2018-05-21 14:55:27,984 [main] INFO  org.apache.hadoop.mapreduce.Job  - Counters: 30
	File System Counters
		FILE: Number of bytes read=0
		FILE: Number of bytes written=3184370
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=1201
		HDFS: Number of bytes written=60820
		HDFS: Number of read operations=40
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=20
	Job Counters 
		Launched map tasks=10
		Other local map tasks=10
		Total time spent by all maps in occupied slots (ms)=139947
		Total time spent by all reduces in occupied slots (ms)=0
		Total time spent by all map tasks (ms)=139947
		Total vcore-seconds taken by all map tasks=139947
		Total megabyte-seconds taken by all map tasks=143305728
	Map-Reduce Framework
		Map input records=206
		Map output records=206
		Input split bytes=1201
		Spilled Records=0
		Failed Shuffles=0
		Merged Map outputs=0
		GC time elapsed (ms)=930
		CPU time spent (ms)=17940
		Physical memory (bytes) snapshot=3550318592
		Virtual memory (bytes) snapshot=16373563392
		Total committed heap usage (bytes)=8241807360
	File Input Format Counters 
		Bytes Read=0
	File Output Format Counters 
		Bytes Written=60820
2018-05-21 14:55:28,013 [main] INFO  org.apache.sqoop.mapreduce.ImportJobBase  - Transferred 59.3945 KB in 49.0563 seconds (1.2107 KB/sec)
2018-05-21 14:55:28,025 [main] INFO  org.apache.sqoop.mapreduce.ImportJobBase  - Retrieved 206 records.
2018-05-21 14:55:28,044 [main] INFO  org.apache.sqoop.tool.ImportTool  - Final destination exists, will run merge job.
2018-05-21 14:55:28,145 [main] INFO  org.apache.hadoop.conf.Configuration.deprecation  - mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
2018-05-21 14:55:28,147 [main] WARN  org.apache.sqoop.mapreduce.JobBase  - SQOOP_HOME is unset. May not be able to find all job dependencies.
2018-05-21 14:55:28,194 [main] INFO  org.apache.hadoop.yarn.client.RMProxy  - Connecting to ResourceManager at ip-172-31-4-192.ap-south-1.compute.internal/172.31.4.192:8032
2018-05-21 14:55:28,393 [main] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Cleaning up the staging area job_1526892304560_0342" target="_blank">/user/hue/.staging/job_1526892304560_0342
2018-05-21 14:55:28,400 [main] WARN  org.apache.hadoop.security.UserGroupInformation  - PriviledgedActionException as:hue (auth:SIMPLE) cause:java.io.FileNotFoundException: File /tmp/sqoop-yarn/compile/8a805c4aae44aa5311448f7cdec9c96f/sales_flat_quote.jar does not exist
2018-05-21 14:55:28,401 [main] ERROR org.apache.sqoop.tool.ImportTool  - Import failed: java.io.FileNotFoundException: File /tmp/sqoop-yarn/compile/8a805c4aae44aa5311448f7cdec9c96f/sales_flat_quote.jar does not exist
	at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:598)
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:811)
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:588)
	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:425)
	at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:340)
	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:2042)
	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:2010)
	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1975)
	at org.apache.hadoop.mapreduce.JobResourceUploader.copyJar(JobResourceUploader.java:210)
	at org.apache.hadoop.mapreduce.JobResourceUploader.uploadFiles(JobResourceUploader.java:166)
	at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:99)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:194)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325)
	at org.apache.sqoop.mapreduce.JobBase.runJob(JobBase.java:403)
	at org.apache.sqoop.mapreduce.MergeJob.runMergeJob(MergeJob.java:157)
	at org.apache.sqoop.tool.ImportTool.lastModifiedMerge(ImportTool.java:459)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:522)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
	at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:243)
	at org.apache.sqoop.tool.JobTool.run(JobTool.java:298)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
	at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:196)
	at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:179)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:60)
	at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:48)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:234)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)


<<< Invocation of Sqoop command completed <<<

Hadoop Job IDs executed by Sqoop: job_1526892304560_0329

Intercepting System.exit(1)

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

Oozie Launcher failed, finishing Hadoop job gracefully

Oozie Launcher, uploading action data to HDFS sequence file: hdfs://ip-172-31-4-192.ap-south-1.compute.internal:8020/user/hue/oozie-oozi/0000011-180521141550131-...
2018-05-21 14:55:28,448 [main] INFO  org.apache.hadoop.io.compress.zlib.ZlibFactory  - Successfully loaded & initialized native-zlib library
2018-05-21 14:55:28,449 [main] INFO  org.apache.hadoop.io.compress.CodecPool  - Got brand-new compressor [.deflate]
Successfully reset security manager from org.apache.oozie.action.hadoop.LauncherSecurityManager@674dc873 to null

Oozie Launcher ends

2018-05-21 14:55:28,582 [main] INFO  org.apache.hadoop.mapred.Task  - Task:attempt_1526892304560_0324_m_000000_0 is done. And is in the process of committing
2018-05-21 14:55:28,582 [main] INFO  org.apache.hadoop.mapred.Task  - Task:attempt_1526892304560_0324_m_000000_0 is done. And is in the process of committing
2018-05-21 14:55:28,628 [main] INFO  org.apache.hadoop.mapred.Task  - Task 'attempt_1526892304560_0324_m_000000_0' done.
2018-05-21 14:55:28,628 [main] INFO  org.apache.hadoop.mapred.Task  - Task 'attempt_1526892304560_0324_m_000000_0' done.
2018-05-21 14:55:28,731 [main] INFO  org.apache.hadoop.metrics2.impl.MetricsSystemImpl  - Stopping MapTask metrics system...
2018-05-21 14:55:28,731 [main] INFO  org.apache.hadoop.metrics2.impl.MetricsSystemImpl  - MapTask metrics system stopped.
2018-05-21 14:55:28,731 [main] INFO  org.apache.hadoop.metrics2.impl.MetricsSystemImpl  - MapTask metrics system shutdown complete.
 

 

 

 

Announcements
New solutions