I am running a mapreduce application in a multi-node cluster, My application is not launching a mapreduce job as its not able to pickup org.apache.hadoop.mapred.YarnClientProtocolProvider as the ClientProtocolProvider - returned null protocol   In return its picking up LocalClientProtocolProvider, where that machine doesn't have all the hadoop services running.   [09 Nov 2017 00:20:20,371] [Thread-0] [op.mapreduce.Cluster] [DEBUG] [Trying ClientProtocolProvider : org.apache.hadoop.mapred.YarnClientProtocolProvider] [09 Nov 2017 00:20:20,371] [Thread-0] [op.mapreduce.Cluster] [DEBUG] [Cannot pick org.apache.hadoop.mapred.YarnClientProtocolProvider as the ClientProtocolProvider - returned null protocol] [09 Nov 2017 00:20:20,371] [Thread-0] [op.mapreduce.Cluster] [DEBUG] [Trying ClientProtocolProvider : org.apache.hadoop.mapred.LocalClientProtocolProvider] [09 Nov 2017 00:20:20,372] [Thread-0] [trics.jvm.JvmMetrics] [INFO] [Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized] [09 Nov 2017 00:20:20,372] [Thread-0] [op.mapreduce.Cluster] [DEBUG] [Picked org.apache.hadoop.mapred.LocalClientProtocolProvider as the ClientProtocolProvider]     [14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,633] [Thread-0] [hadoop.mapreduce.Job] [INFO] [ map 100% reduce 0%] [14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,633] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.getTaskCompletionEvents(Job.java:687)] [14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,633] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)] [14 Nov 2017 21:09:12,633] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,634] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)] [14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,634] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.getTaskCompletionEvents(Job.java:687)] [14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,634] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)] [14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,634] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)] [14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,634] [Thread-0] [hadoop.mapreduce.Job] [INFO] [Job job_local1205960386_0001 failed with state FAILED due to: NA] [14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,634] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,635] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.getCounters(Job.java:775)] [14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,635] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,653] [Thread-0] [hadoop.mapreduce.Job] [INFO] [Counters: 29 File System Counters FILE: Number of bytes read=864751596 FILE: Number of bytes written=1697573454 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=24030843 HDFS: Number of bytes written=0 HDFS: Number of read operations=9 HDFS: Number of large read operations=0 HDFS: Number of write operations=0 Map-Reduce Framework Map input records=87587 Map output records=87587 Map output bytes=831780692 Map output materialized bytes=832131046 Input split bytes=383 Combine input records=0 Spilled Records=175174 Failed Shuffles=0 Merged Map outputs=0 GC time elapsed (ms)=164 CPU time spent (ms)=23130 Physical memory (bytes) snapshot=1342242816 Virtual memory (bytes) snapshot=4710432768 Total committed heap usage (bytes)=1676673024 applicationxxxx total-map-time-millis=9719 File Input Format Counters Bytes Read=0 parquet bytesread=24019232 bytestotal=24019232 timeread=327] [14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,653] [Thread-0] [UserGroupInformation] [DEBUG] [PrivilegedAction as:xyz@xyzCLUSTER.COM (auth:KERBEROS) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:323)] [14 Nov 2017 21:09:12,653] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,654] [Thread-0] [op.GroupMapperDriver] [INFO] [ExistStatus : 0] [14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0] [14 Nov 2017 21:09:12,654] [main] [xxx.yyy.zzzz.qqqq] [INFO] [applinationxxx: Shutting down...] [14 Nov 2017 21:09:12,654] [localfetcher#1] [.reduce.LocalFetcher] [DEBUG] [LocalFetcher 1 going to fetch: attempt_local1205960386_0001_m_000000_0]   How do we set application to use YarnClientProvider instead of LocalclientProtocolProvider. ?? After Map has finsihed 100% reduce is never started , why is that ?? Any help or sugestions would be appreciated. Thanks ... View more