Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Difference between HADOOP and YARN job

Highlighted

Difference between HADOOP and YARN job

Expert Contributor

Folks,

this may be a simple question. what is difference between the below two :

1) yarn jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar pi 10 10000;

2) hadoop jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar pi 10 10000;

I did run both and compared the logs, still same. In fact both took same amount of time.

Thanks

Kumar

4 REPLIES 4
Highlighted

Re: Difference between HADOOP and YARN job

Al you are doing is running MR code through yarn and hadoop so it looks the same.

Please try starting a spark job from hadoop jar. hadoop job is a subset of Yarn jobs.

Highlighted

Re: Difference between HADOOP and YARN job

Super Collaborator

But you use spark-submit for a Spark Job, not "hadoop/yarn jar"

Highlighted

Re: Difference between HADOOP and YARN job

spark-submit has YARN client wrapped within it, as spark-submit when executed in YARN mode, request yarn to start an AM and then request for containers where the SPARK ETL is executed.

Highlighted

Re: Difference between HADOOP and YARN job

Super Collaborator
Don't have an account?
Coming from Hortonworks? Activate your account here