question Re: Java 8 on CDH cluster using Cloudera director (AWS) in Archives of Support Questions (Read Only)

Java 8 on CDH cluster using Cloudera director (AWS)

sunilcsc — Fri, 16 Sep 2022 10:28:51 GMT

We require java 8 on the CDH cluster to support analytics tools so Cloudera’s suggestion to how to solve this would be very helpful at this point. We have a CDH cluster on AWS that was spun up using Cloudera Director. Unfortunately, that spun up with java 1.6 and 1.7 as Cloudera director automatically installs and points to 1.6

The upgrade to 1.8.0_60 isn’t going so smooth. We are going to put that aside for the moment and look to get a cluster spun up with 1.8.0_60 from the start. Is there a suggested path from Cloudera on this? We want to use Cloudera Director to handle the installs and looking for an official Cloudera response as to the use of java 8 as ‘the’ java to use for the cluster.

Cloudera’s docs on java 8 upgrade are abstracted out to a high level so obviously there are issues to get past. The issue I am seeing currently is on a data node and trying to spin it up after updating to 1.8.0_60. Screenshot attached.

Re: Java 8 on CDH cluster using Cloudera director (AWS)

dhan — Thu, 07 Jul 2016 14:19:48 GMT

sunilcsc,

It is possible to create a cluster with java 8 through Director.

The Deployment (CM) must be created with javaInstallationStrategy set to NONE. This setting tells CM not to install java on the CDH hosts. This property is not exposed through the UI so you need to create the cluster using a conf file.

...
cloudera-manager {
   ...
   javaInstallationStrategy: NONE
   ...
}
...

Java 8 needs to be installed on each instance using a bootstrap script.

Here is an example bootstrap script you can use as a starting point. It removes openjdk, downloads a java 8 rpm, and installs it.

#!/bin/sh

yum remove --assumeyes *openjdk*
yum install --assumeyes wget
cd /opt
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u45-b14/jdk-8u45-linux-x64.rpm"
yum install --assumeyes /opt/jdk-8u45-linux-x64.rpm

Re: Java 8 on CDH cluster using Cloudera director (AWS)

Mike Wilson — Thu, 07 Jul 2016 16:27:01 GMT

It should be noted that the structure of this script works, but I'd advise you to use a more stable location for that RPM than Oracle's site. Oracle only allows you to download the 2 most recent versions of Java 8 without logging in, which this script does not do. Java 8u45 likely requires this login to download. You could use something like an internal S3 bucket (provided you're on AWS) to host it and download it internally.

Re: Java 8 on CDH cluster using Cloudera director (AWS)

gardella — Fri, 08 Jul 2016 04:44:42 GMT

Perfect. Exactly what I was looking for. Thank you!

I will take and put java in an S3 bucket as suggested below as well.

Many thanks!

Re: Java 8 on CDH cluster using Cloudera director (AWS)

mjarnold — Sat, 03 Sep 2016 00:51:46 GMT

When will Director support this natively?

Re: Java 8 on CDH cluster using Cloudera director (AWS)

Vinithra — Sat, 03 Sep 2016 01:17:51 GMT

Hi, we currently don't have a scheduled date for this, but we will take
your request into consideration during prioritization.

Re: Java 8 on CDH cluster using Cloudera director (AWS)

Tomas79 — Thu, 20 Jul 2017 11:50:17 GMT

I wanted to start a new thread but having very similiar question. Why is Cloudera Manager / Director deploying java 1.6 when Cloudera clearly states that the lowest possible supported version is 1.7?

Thanks for the clarification

https://www.cloudera.com/documentation/enterprise/5-4-x/topics/cdh_ig_req_supported_versions.html#concept_pdd_kzf_vp

Re: Java 8 on CDH cluster using Cloudera director (AWS)

Mike Wilson — Thu, 20 Jul 2017 20:26:11 GMT

Director installs Java 7 by default and has since version 1.0. Is it possible that Java is unexpectedly installed on the instance type that you're using? Director attempts to remove existing Java versions prior to installing its own version, but occasionally certain images have Java installed in ways that Director isn't anticipating.

What version of Director are you using? Assuming that you're using AWS: what region and what AMI are you using to try to bootstrap the cluster?

Re: Java 8 on CDH cluster using Cloudera director (AWS)

Tomas79 — Fri, 21 Jul 2017 09:45:14 GMT

The latest version of Director (2.4 I assume).
I really tooked a simple wizard, went throught with 1 master, 3 workers, 1 gateway, installed the cluster and checked the java in every host (1.6)
AMI Centos 7, tried region eu-west-1, us-east-1.

Re: Java 8 on CDH cluster using Cloudera director (AWS)

Mike Wilson — Mon, 24 Jul 2017 15:23:41 GMT

Can you give me the exact AMI IDs that you are using to try out?