MR2 job is not triggered, YarnClientProtocolProvider returned null

Shyam2131 — Fri, 16 Sep 2022 12:32:27 GMT

I am running a mapreduce application in a multi-node cluster, My application is not launching a mapreduce job as its not able to pickup org.apache.hadoop.mapred.YarnClientProtocolProvider as the ClientProtocolProvider - returned null protocol

In return its picking up LocalClientProtocolProvider, where that machine doesn't have all the hadoop services running.

[09 Nov 2017 00:20:20,371] [Thread-0] [op.mapreduce.Cluster] [DEBUG] [Trying ClientProtocolProvider : org.apache.hadoop.mapred.YarnClientProtocolProvider] [09 Nov 2017 00:20:20,371] [Thread-0] [op.mapreduce.Cluster] [DEBUG] [Cannot pick org.apache.hadoop.mapred.YarnClientProtocolProvider as the ClientProtocolProvider - returned null protocol] [09 Nov 2017 00:20:20,371] [Thread-0] [op.mapreduce.Cluster] [DEBUG] [Trying ClientProtocolProvider : org.apache.hadoop.mapred.LocalClientProtocolProvider] [09 Nov 2017 00:20:20,372] [Thread-0] [trics.jvm.JvmMetrics] [INFO] [Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized] [09 Nov 2017 00:20:20,372] [Thread-0] [op.mapreduce.Cluster] [DEBUG] [Picked org.apache.hadoop.mapred.LocalClientProtocolProvider as the ClientProtocolProvider]

[14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,633] [Thread-0] [hadoop.mapreduce.Job] [INFO] [ map 100% reduce 0%]
[14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,633] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.getTaskCompletionEvents(Job.java:687)]
[14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,633] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)]
[14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,634] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)]
[14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,634] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.getTaskCompletionEvents(Job.java:687)]
[14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,634] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)]
[14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,634] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)]
[14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,634] [Thread-0] [hadoop.mapreduce.Job] [INFO] [Job job_local1205960386_0001 failed with state FAILED due to: NA]
[14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,635] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.getCounters(Job.java:775)]
[14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,653] [Thread-0] [hadoop.mapreduce.Job] [INFO] [Counters: 29
File System Counters
FILE: Number of bytes read=864751596
FILE: Number of bytes written=1697573454
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=24030843
HDFS: Number of bytes written=0
HDFS: Number of read operations=9
HDFS: Number of large read operations=0
HDFS: Number of write operations=0
Map-Reduce Framework
Map input records=87587
Map output records=87587
Map output bytes=831780692
Map output materialized bytes=832131046
Input split bytes=383
Combine input records=0
Spilled Records=175174
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=164
CPU time spent (ms)=23130
Physical memory (bytes) snapshot=1342242816
Virtual memory (bytes) snapshot=4710432768
Total committed heap usage (bytes)=1676673024
applicationxxxx
total-map-time-millis=9719
File Input Format Counters
Bytes Read=0
parquet
bytesread=24019232
bytestotal=24019232
timeread=327]
[14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,653] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)]
[14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,654] [Thread-0] [op.GroupMapperDriver] [INFO] [ExistStatus : 0]
[14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]
[14 Nov 2017 21:09:12,654] [main] [xxx.yyy.zzzz.qqqq] [INFO] [applinationxxx: Shutting down...]
[14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]

How do we set application to use YarnClientProvider instead of LocalclientProtocolProvider. ??

After Map has finsihed 100% reduce is never started , why is that ??

Any help or sugestions would be appreciated.

Thanks

Re: MR2 job is not triggered, YarnClientProtocolProvider returned null

Shyam2131 — Fri, 01 Dec 2017 16:09:44 GMT

I figured out what the issue was. My configuration object was not able to get mapreduce.framework.name as "yarn" from mapred-site.xml. So I was overiding mapreduce.framework.name in driver. So that it can pick YarnClientProtocolProvider instead of LocalClientProtocolProvider.

When yarn is picked up it launched mapreduce job.

question Re: MR2 job is not triggered, YarnClientProtocolProvider returned null in Support Questions

MR2 job is not triggered, YarnClientProtocolProvider returned null

Re: MR2 job is not triggered, YarnClientProtocolProvider returned null