Reply
Highlighted
Explorer
Posts: 23
Registered: ‎01-14-2015

Change JAVA version for Spark on YARN

I have installed JAVA 8 on all the servers and changed JAVA HOME DIRECTORY for hosts to point to JAVA 8. But when I start spark-shell it always used JAVA 7. Is there a way to force it to use java 8?

 

Posts: 1,836
Kudos: 416
Solutions: 295
Registered: ‎07-31-2013

Re: Change JAVA version for Spark on YARN

The CM configuration for JAVA HOME (under CM -> Hosts -> Configuration page) does not cover client tools and commands currently.

 

You will need an explicit JAVA_HOME in your environment pointing to Java 8, or the symlink of /usr/java/latest and /usr/java/default pointing to Java 8 home directories to make the client programs run the newer JDK.

 

Example:

~> JAVA_HOME=/usr/java/jdk1.8.0_66/ spark-shell