Support Questions

Find answers, ask questions, and share your expertise

HBASE_HOME path no longer in YARN classpath?


We are currently working on testing some of our older workflows from hadoop 4.x in 6.x (yes large jump)

and we have a particular phase of a workflow fail which seems to have to do with the classpath. We now in our 4.x cluster that the classpath for yarn in 4.x shows as:



4.x yarn classpath:


yet in 6.3.2 it shows as:


6.3.2 classpath for yarn


 so when our workflow is ran it 6.x it fails with:


java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/util/Bytes


But,  if I manually the phase that's failing

java -cp my-jar.jar:`hbase classpath` MyMain my args
It works fine.  If instead of hbase classpath I use hadoop classpath, I get the exact same error:
java -cp my-jar.jar:`hadoop classpath` MyMain my args
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/util/Bytes
So, we know it's the classpath.  But we haven't found a way for oozie/yarn to send us the right classpath.
We have tried updating the classpath.txt file as a test and it still fails.  We have also tried updating the classpath for yarn and yet again fails.  So where in the world do we add our classpath in 6.x that is needed?
NOTE;  We don't bundle any of our jars so we knows its not that.  
I know I've seen articles about having to add this to HADOOP_OPTS, but we don't know where that is in cdh 6.x.  Can someone please point us in the right direction or give us a helping hand?  I can post logs and debug info if needed (thus the reason we are trying this on 6.3.x 🙂 🙂 ..  




In 6.3.x are we needing to set something in  oozie.service.HadoopAccessorService.hadoop.configurations   

just trying to get to the bottom of this.  


Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.