Support Questions

Find answers, ask questions, and share your expertise

java.io.FileNotFoundException: File does not exist: hdfs://ABC.xyz.net:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.2-mr-framework.tar.gz

New Contributor

I was testing if mapreduce is working properly on my cloudera 6.3, CDH 6.3.2 cluster.

I have installed cloudera using one-click-installer Proof-of-concept one!

 

while i excecuted the below command as it is in this link

https://docs.cloudera.com/documentation/other/tutorial/CDH5/topics/ht_usage.html

 

 

Command:

sudo -u hdfs hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 100

 

i Got below error:

 

WARNING: Use "yarn jar" to launch YARN applications.
Number of Maps = 10
Samples per Map = 100
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Starting Job
20/01/02 15:18:01 INFO client.RMProxy: Connecting to ResourceManager at ABC.xyz.net/192.168.xx.xx:8032
java.io.FileNotFoundException: File does not exist: hdfs://ABC.xyz.net:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.2-mr-framework.tar.gz
at org.apache.hadoop.fs.Hdfs.getFileStatus(Hdfs.java:145)
at org.apache.hadoop.fs.AbstractFileSystem.resolvePath(AbstractFileSystem.java:488)
at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2225)
at org.apache.hadoop.fs.FileContext$25.next(FileContext.java:2221)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.resolve(FileContext.java:2227)
at org.apache.hadoop.fs.FileContext.resolvePath(FileContext.java:607)
at org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(JobSubmitter.java:460)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:146)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:307)
at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:360)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:368)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

 

 

Please help me understand the issue.

Thanks for the reply.
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:313)
at org.apache.hadoop.util.RunJar.main(RunJar.java:227)

2 ACCEPTED SOLUTIONS

Guru
@backlashhardik ,

Please try below:

1. update HDFS path permissions to make sure:

hdfs dfs -chmod 755 /user/yarn
hdfs dfs -chmod 755 /user/yarn/mapreduce

you might need to run as "hdfs" user

2. Then install the Yarn MapReduce Frame jar
CM -> YARN -> Actions -> Install YARN MapReduce frame jar

Please let me know if that helps.

Cheers
Eric

View solution in original post

Guru
Sorry, forgot to mention that you might need to stop YARN service first before step 2.

Cheers
Eric

View solution in original post

9 REPLIES 9

Guru
@backlashhardik ,

Please try below:

1. update HDFS path permissions to make sure:

hdfs dfs -chmod 755 /user/yarn
hdfs dfs -chmod 755 /user/yarn/mapreduce

you might need to run as "hdfs" user

2. Then install the Yarn MapReduce Frame jar
CM -> YARN -> Actions -> Install YARN MapReduce frame jar

Please let me know if that helps.

Cheers
Eric

New Contributor

It worked for me.

Thanks

New Contributor

solution worked for me. Thank you

New Contributor

Thank you.

 

Solution worked for me.

New Contributor

hi i have similiar problem but it is kafka  instead of mapr. would you mind to check kafka to hdfs file not found exception 

Guru
Sorry, forgot to mention that you might need to stop YARN service first before step 2.

Cheers
Eric

New Contributor

WORKED FOR ME PREFECTLY!

Thanks for identifying it so soon!

@EricL 🙂

Guru
No problem, glad that it helped.

Cheers
Eric

Explorer

Well done! it worked for me.

------------------------------------

and from my test... 

just this step CM -> YARN -> Actions -> Install YARN MapReduce frame jar  enough.

nothing else. CM will do else for u automatically.

 

from CDH 6.2

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.