HDP installation via Ambari - a doubt about the jdk and Internet connection

[#1]

Machines : (4 datanodes + 2 master(HA) + 1 management = 7 machines) .

Target : Install Ambari 2.2 and using it, install HDP 2.4(the auto. install way).

The Ambari and HDP repositories are available locally(on the management node, tarballs were extracted) via http.

Ambari server is already running on the management machine and now the HDP 2.4 has to be installed.

Questions :

1. To avoid installing JDK on the management(and others, too) machine, the Internet access to http://public-repo-1.hortonworks.com/ has been enabled on all the machines only for a day. I set the 'export http_proxy' and set-up the Ambari server which internally fetched the Oracle 8 JDK. Somehow, 'java -version' still doesn't work, does Ambari really install JDK
2. The Ambari agents will be installed auto. during the cluster install later but then there would be NO Internet connection. How is java(jdk) installed on the other nodes then ? Is it that Ambari pushes the /var/lib/ambari-server/resources/jdk-8u60-linux-x64.tar.gz to all the nodes manually(after all, hadoop would require java)
3. Is it safe to remove the Internet access now

[#2]

Kaliyug Antagonist

Q. To avoid installing JDK on the management(and others, too) machine, the Internet access to http://public-repo-1.hortonworks.com/ has been enabled on all the machines only for a day. I set the 'export http_proxy' and set-up the Ambari server which internally fetched the Oracle 8 JDK. Somehow, 'java -version' still doesn't work, does Ambari really install JDK.

Ans. Have you set JAVA_HOME correctly? Ambari by default install java at /usr/jdk64

Q. The Ambari agents will be installed auto. during the cluster install later but then there would be NO Internet connection. How is java(jdk) installed on the other nodes then ? Is it that Ambari pushes the /var/lib/ambari-server/resources/jdk-8u60-linux-x64.tar.gz to all the nodes manually(after all, hadoop would require java)

Ans. Yes that is correct.

Q. Is it safe to remove the Internet access now

Ans. Yes it is safe. However not sure of you have other dependencies on internet.

[#3]

• Have you set JAVA_HOME correctly? Ambari by default install java at /usr/jdk64

Yes, the jdk 1.8 exists at /usr/jdk64 but I assumed that Ambari sets the JAVA_HOME because if one selects the 'Custom JDK' option during ambari server set-up, it prompts for providing the JAVA_HOME. I am just wondering how Ambari accesses Java

• However not sure of you have other dependencies on internet.

If the repositories and JDK is now available locally, will Ambari still try to access the Internet? Can you elaborate on the other dependencies ?

[#4 · Accepted solution]

@Kaliyug Antagonist

Q. If the repositories and JDK is now available locally, will Ambari still try to access the Internet? Can you elaborate on the other dependencies ?

Ans. Your ambari server will not need internet connection if you are not adding it to cluster hosts.

However for all the hosts which you will register by ambari, assuming you are using centos or redhat server; when you try to register hosts it will do yum install for hadoop packages and there may be certain dependencies for that packages which will be fetched from Centos base repo.

For installing those dependencies you will need internet or you have to create local base repo of centos.

[#5]

@Kaliyug Antagonist

Setup a Local Repository in this scenario. Refer to Ambari Install guide for the same.