Java 8 on CDH with Cloudera Director 2.7.0 (AWS)

[#1]

Hello,

I have a problem when I try to deploy a cluster with Java 8 through Cloudera Director, with the 2.7.0 version running on AWS EC2.

I tried to follow the documentation, but I got some issues : 

-  cmJavaPackages and defaultCmJavaPackage properties are missing in the application.properties file. Instead, there is a property named lp.bootstrap.packages.javaPackage. 
Here is what I put in order to have Java 8: 

lp.bootstrap.packages.javaPackage: oracle-j2sdk1.8
lp.bootstrap.packages.cmJavaPackages: .*=oracle-j2sdk1.8
lp.bootstrap.packages.defaultCmJavaPackage: oracle-j2sdk1.8

- Then, when I create a Cloudera Manager instance with the Director UI (with adding the Cloudera Director yum repo in the instance bootstraping), I have both Java 7 and Java 8 JDKs installed on the instance

ls /usr/java
jdk1.7.0_67-cloudera jdk1.8.0_121-cloudera

- Finally, when I launch a cluster (so with the AUTO JDK Installation Strategy), I only have Java 7 JDK on my instances.

I use the last RHEL AMI (RHEL-7.4_HVM-20180103-x86_64-2-Hourly2-GP2 (ami-194cdc76)) on my instances, and this bootstrap script : 

sudo yum install --assumeyes wget
cd /etc/yum.repos.d/
sudo wget "https://archive.cloudera.com/director/redhat/7/x86_64/director/cloudera-director.repo"

Can you help me ?
Did I miss something important in the configuration ?

Thanks a lot

[#2 · Accepted solution]

Hi Behel,

Yes, we didn't get the example application.properties values correct in the 2.7.0 release, but the cmJavaPackages and defaultCmJavaPackages are the correct ones to set. cmJavaPackages is a list, so the syntax is a little odd (and our docs might not be quite right, still):

lp.bootstrap.packages.cmJavaPackages[0]: ".*=oracle-j2sdk1.8"

lp.bootstrap.packages.defaultCmJavaPackage: oracle-j2sdk1.8

The "javaPackage" property doesn't have any effect, so you can remove it.

When you use the UI, Cloudera Director uses the AUTO Java installation strategy, so the settings won't have any effect on the cluster instances, just on the CM instance. Under AUTO, Director lets Cloudera Manager handle the cluster instances, and they only get Java 7 and maybe Java 6, depending on the version.

To get just Java 8 on everything, use the DIRECTOR_MANAGED Java installation strategy, which requires you to use a client configuration file.

Your setup of the Cloudera Director repo to make the oracle-j2sdk1.8 package available looks correct to me. You can always test it out manually by running yum from the command line.

Do let me know if the above helps you get things working!

Bill

[#3]

Thank you very much.

It doesn't work with lp.bootstrap.packages.cmJavaPackages[0]: ".*=oracle-j2sdk1.8" as a property, I have this error in the logs when I try to use the CLI : 

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'lp.bootstrap.packages-com.cloudera.launchpad.bootstrap.PackageIndex$ConfigProperties': Could not bind properties to PackageIndex.ConfigProperties (prefix=lp.bootstrap.packages, ignoreInvalidFields=false, ignoreUnknownFields=true, ignoreNestedProperties=false); nested exception is java.lang.UnsupportedOperationException

So I put again lp.bootstrap.packages.cmJavaPackages: .*=oracle-j2sdk1.8 as property and it works fine.

I misunderstood the Auto Strategy, maybe the documentation isn't very clear about it. I thought that this strategy forced Manager to deploy Java 8 on the cluster instances.

It's ok with the Director_Managed Strategy, I have just Java 8 on my instances 

Thanks a lot !