SYMPTOM: The MR classpath generated in HDP 2.2.0+ no longer includes the Hadoop config files that were present in previous versions. Signs of this include an inability to read core-site.xml and find properties such as fs.defaultFS, after upgrading from HDP 2.1.x or earlier. Error messages in stdout may look similar to: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService ROOT CAUSE: As of HDP 2.2.0, the classpath is handled differently, in that MapReduce does not include the cluster classpath. This is intended to isolate a running MapReduce application from jar version changes during a rolling upgrade scenario. WORKAROUND: Potential workarounds include: Change code to add the required classpath elements explicitly. Change code to exec the hadoop classpath shell command to determine the cluster classpath at runtime. If code cannot be changed, reconfigure mapreduce.application.classpath in mapred-site.xml so that it does include the cluster classpath. However, this can compromise classpath isolation for running MapReduce jobs during a future rolling upgrade, and is therefore not recommended. Example: Launch the application using a shell script that includes export CLASSPATH=$CLASSPATH:`hadoop classpath` RESOLUTION: Working as designed. Use one of the above workarounds if necessary. ... View more