WARNING: Use "yarn jar" to launch YARN applications

[#1]

Hi:

When I submit the job from RStudio and Pig, i can see that:

WARNING: Use "yarn jar" to launch YARN applications.

Its is because the Manager has the yarn and mapreduce2 installed????

Thanks

[#2]

@Roberto Sancho

If your job is finishing then I would ignore that warning .

[#3]

Hi:

Thanks, but its not clearly for me, the problem is from HDP cluster, or RStudio classpath.

How can i redirect the steout??

Thanks

[#4]

message is from HDP, you can ignore it. I highly recommend to create an article and show step by step your work with RStudio and Pig. A lot of people would be interested. @Roberto Sancho

[#5]

Just a bit explanation in MapReduce1 there was the "hadoop" command to administrate your cluster ( run mapreduce programs etc. ). However Yarn comes with its own command line command for administration "yarn". Pig is under the cover using "hadoop jar" to run its compiled MapReduce program while HDP would like end users to use the newer "yarn jar". That is the warning.

However "hadoop jar" is perfectly fine and if it ever would be deprecated it would be updated in pig as well. So yes you can safely ignore this warning.

[#6]

HI:

Ok ill, do it, where i can put the steps or de documentations??

Thanks

[#7 · Accepted solution]

Hi:

Today I have resolved this with those code from R

Sys.setenv("HADOOP_CMD"="/usr/hdp/2.3.2.0-2950/hadoop/bin/yarn")
Sys.setenv("HADOOP_STREAMING"="/usr/hdp/2.3.2.0-2950/hadoop-mapreduce/hadoop-streaming-2.7.1.2.3.2.0-2950.jar")
Sys.setenv("HADOOP_HOME"="/usr/hdp/2.3.2.0-2950/hadoop")
Sys.setenv("HADOOP_PREFIX"="/usr/hdp/2.3.2.0-2950/hadoop")