https://community.cloudera.com/t5/Archives-of-Support-Questions/Configuration-objects-fail-to-initialize-in-HDP-2-2-4-8/m-p/95928#M9288 <P><STRONG>SYMPTOM</STRONG>: The MR classpath generated in HDP 2.2.0+ no longer includes the Hadoop config files that were present in previous versions. Signs of this include an inability to read core-site.xml and find properties such as fs.defaultFS, after upgrading from HDP 2.1.x or earlier. Error messages in stdout may look similar to:</P><P>Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService </P><STRONG>ROOT CAUSE</STRONG>: As of HDP 2.2.0, the classpath is handled differently, in that MapReduce does not include the cluster classpath. This is intended to isolate a running MapReduce application from jar version changes during a rolling upgrade scenario.<STRONG> </STRONG> <STRONG>WORKAROUND</STRONG>: Potential workarounds include: <OL> <LI>Change code to add the required classpath elements explicitly.</LI><LI>Change code to exec the hadoop classpath shell command to determine the cluster classpath at runtime.</LI><LI>If code cannot be changed, reconfigure mapreduce.application.classpath in mapred-site.xml so that it does include the cluster classpath. However, this can compromise classpath isolation for running MapReduce jobs during a future rolling upgrade, and is therefore not recommended.</LI></OL><STRONG>Example:</STRONG> Launch the application using a shell script that includes<PRE>export CLASSPATH=$CLASSPATH:`hadoop classpath`</PRE><STRONG>RESOLUTION</STRONG>: Working as designed. Use one of the above workarounds if necessary. Fri, 23 Oct 2015 21:06:03 GMT amiller 2015-10-23T21:06:03Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Configuration-objects-fail-to-initialize-in-HDP-2-2-4-8/m-p/95927#M9287 <P>In order to facilitate rolling upgrades, /etc/hadoop/conf is not part of the CLASSPATH env var that is constructed for a job. However when jobs are using a Configuration object: Configuration conf2 = new Configuration(); <A href="https://hadoop.apache.org/docs/current/api/org/apache/hadoop/conf/Configuration.html">https://hadoop.apache.org/docs/current/api/org/apache/hadoop/conf/Configuration.html</A> It tries to read the core-site.xml from the CLASSPATH</P><P>How can this be addressed?</P> Fri, 23 Oct 2015 16:18:03 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Configuration-objects-fail-to-initialize-in-HDP-2-2-4-8/m-p/95927#M9287 edemauro 2015-10-23T16:18:03Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Configuration-objects-fail-to-initialize-in-HDP-2-2-4-8/m-p/95928#M9288 <P><STRONG>SYMPTOM</STRONG>: The MR classpath generated in HDP 2.2.0+ no longer includes the Hadoop config files that were present in previous versions. Signs of this include an inability to read core-site.xml and find properties such as fs.defaultFS, after upgrading from HDP 2.1.x or earlier. Error messages in stdout may look similar to:</P><P>Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService </P><STRONG>ROOT CAUSE</STRONG>: As of HDP 2.2.0, the classpath is handled differently, in that MapReduce does not include the cluster classpath. This is intended to isolate a running MapReduce application from jar version changes during a rolling upgrade scenario.<STRONG> </STRONG> <STRONG>WORKAROUND</STRONG>: Potential workarounds include: <OL> <LI>Change code to add the required classpath elements explicitly.</LI><LI>Change code to exec the hadoop classpath shell command to determine the cluster classpath at runtime.</LI><LI>If code cannot be changed, reconfigure mapreduce.application.classpath in mapred-site.xml so that it does include the cluster classpath. However, this can compromise classpath isolation for running MapReduce jobs during a future rolling upgrade, and is therefore not recommended.</LI></OL><STRONG>Example:</STRONG> Launch the application using a shell script that includes<PRE>export CLASSPATH=$CLASSPATH:`hadoop classpath`</PRE><STRONG>RESOLUTION</STRONG>: Working as designed. Use one of the above workarounds if necessary. Fri, 23 Oct 2015 21:06:03 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Configuration-objects-fail-to-initialize-in-HDP-2-2-4-8/m-p/95928#M9288 amiller 2015-10-23T21:06:03Z