CDP Cloudera Manager Fails to install Cloudera Runtime

[#1]

Hello,

I tried to install a trial CDP Cluster following the given steps: https://docs.cloudera.com/cdp-private-cloud-base/latest/installation/topics/cdpdc-trial-installation...

The Cloudera Manager executed is https://archive.cloudera.com/cm7/7.4.4/cloudera-manager-installer.bin It was installed successfully.

The Cluster installation is failing with the messages:

Uninstalled. View Failed to install openjdk8 package. 
Error: Cannot find a valid baseurl for repo: cloudera-manager

This is the complete log:

Uninstalled. View Failed to install openjdk8 package. View

/tmp/scm_prepare_node.pCEw7JVQ
using SSH_CLIENT to get the SCM hostname: 100.200.196.179 59194 22
opening logging file descriptor
Starting installation script...
Acquiring installation lock...
BEGIN flock 4
END (0)
Detecting root privileges...
effective UID is 1000
BEGIN which pbrun
END (1)
BEGIN sudo -S id
/usr/bin/which: no pbrun in (/home/opc/.local/bin:/home/opc/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin)
END (0)
Using 'sudo ' to acquire root privileges
Detecting distribution...
BEGIN grep Tikanga /etc/redhat-release
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
END (1)
BEGIN grep 'Scientific Linux release 5' /etc/redhat-release
END (1)
BEGIN grep Santiago /etc/redhat-release
END (1)
BEGIN grep 'CentOS Linux release 6' /etc/redhat-release
END (1)
BEGIN grep 'CentOS release 6' /etc/redhat-release
END (1)
BEGIN grep 'Scientific Linux release 6' /etc/redhat-release
END (1)
BEGIN grep Maipo /etc/redhat-release
END (1)
BEGIN grep Ootpa /etc/redhat-release
END (0)
/etc/redhat-release ==> RHEL 8
Detecting Cloudera Manager Server...
BEGIN sudo yum -y install bind-utils
Red Hat Enterprise Linux release 8.9 (Ootpa)
Last metadata expiration check: 0:01:32 ago on Tue 23 Apr 2024 11:06:53 PM GMT.
Package bind-utils-32:9.11.36-11.el8_9.1.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
END (0)
BEGIN host -t PTR 100.200.196.179
189.196.111.100.in-addr.arpa domain name pointer ml.phx.personal.com.
END (0)
using ml.phx.personal.com as scm server hostname
BEGIN which python
/usr/bin/python
END (0)
BEGIN python -c 'import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect((sys.argv[1], int(sys.argv[2]))); s.close();' ml.phx.personal.com 7182
END (0)
BEGIN which wget
END (0)
/usr/bin/wget
BEGIN wget -qO- -T 1 -t 1 http://169.254.169.254/latest/meta-data/public-hostname && /bin/echo
END (8)
Installing package repositories...
Checking https://archive.cloudera.com/cm7/7.4.4/redhat8/yum/repodata/repomd.xml
Checking https://archive.cloudera.com/cm7/7.4.4/repodata/repomd.xml
Using
installing repository file /tmp/scm_prepare_node.pCEw7JVQ/repos/rhel8/cloudera-manager.repo
repository file /tmp/scm_prepare_node.pCEw7JVQ/repos/rhel8/cloudera-manager.repo installed
installing rpm keys
BEGIN gpg --import /tmp/scm_prepare_node.pCEw7JVQ/customGPG
gpg: keybox '/tmp/scm_prepare_node.pCEw7JVQ/gnupg.BgYX5lHFzp/pubring.kbx' created
gpg: /tmp/scm_prepare_node.pCEw7JVQ/gnupg.BgYX5lHFzp/trustdb.gpg: trustdb created
gpg: key 327574EE02A818DD: public key "Cloudera Apt Repository" imported
gpg: key F90C0D8FE8F86ACD: public key "Yum Maintainer <webmaster@cloudera.com>" imported
gpg: key 73985D43B0B19C9F: public key "Parameterized Build <security@cloudera.com>" imported
gpg: key 2279662784415700: public key "Cloudera <security@cloudera.com>" imported
gpg: key 0F65552736F57F35: public key "Cloudera <security@cloudera.com>" imported
gpg: Total number processed: 5
gpg: imported: 5
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/F36A89E33CC1BD0F71079007327574EE02A818DD.pub
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/12B52DEFA56D22DCBCB471FF84080586D1CA74A1.pub
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/5F14D39EF0681ACA6F044A43F90C0D8FE8F86ACD.pub
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/7B190376A6113C5C2549955B11BA70F151184508.pub
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/9543951160C284C0E7CA254573985D43B0B19C9F.pub
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/E5DFAD1E19FDE5DEF3D3F6848839DC26A96421E4.pub
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/CECDB80C4E9004B0CFE852962279662784415700.pub
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/0103D08AC48124C3ACDF729D74EC0834205B70ED.pub
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/DF2C4DD7629B1AC08A0966E00F65552736F57F35.pub
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/221AB118CB3531ECAAC5852455D3E5536180AD2B.pub
END (0)
BEGIN sudo rpm --import /tmp/scm_prepare_node.pCEw7JVQ/FA92354299AF0F5C3056AE2FD7EC5050A59875AF.pub
END (0)
Refreshing package metadata...
BEGIN sudo yum --disablerepo=* --enablerepo=cloudera* clean all
74 files removed
END (0)
BEGIN sudo rm -Rf /var/cache/yum/*
END (0)
BEGIN sudo yum --disablerepo=* --enablerepo=cloudera* makecache
Error: Cannot find a valid baseurl for repo: cloudera-manager
END (1)
Installing openjdk8 package...
BEGIN sudo yum --disablerepo=* --enablerepo=cloudera* list installed openjdk8
Installed Packages
openjdk8.x86_64 8.0+232_9-cloudera @Cloudera-manager
END (0)
BEGIN echo openjdk8 cloudera-manager-agent cloudera-manager-daemons | grep openjdk8
openjdk8 cloudera-manager-agent cloudera-manager-daemons
END (0)
BEGIN sudo yum --disablerepo=* --enablerepo=cloudera* info openjdk8
Error: Cannot find a valid baseurl for repo: cloudera-manager
END (1)
remote package openjdk8 is not available, giving up
waiting for rollback request
detected rollback request
rolling back installation
Reverting changes...
rollback started
Removing package repositories...
repository file /etc/yum.repos.d/cloudera-manager.repo restored
Cleaning the package manager cache...
BEGIN sudo yum --disablerepo=* --enablerepo=cloudera* clean all
0 files removed
END (0)
BEGIN sudo rm -Rf /var/cache/yum/*
END (0)
Uninstalled.
rollback completed
closing logging file descriptor

The commands that are failing like sudo yum --disablerepo=* --enablerepo=cloudera* info openjdk8 were executed in a terminal without issues.

Any Idea what is missing on this Trial Installation? 

[#2]

@Marks_08 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our CDP experts @abdulpasithali @venkatsambath  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3]

Hello @Marks_08 

Thank you for reaching out 

Firstly please confirm whether are you using the local repository for the installation of CDP. If yes are the required packages present like openjdk and cm packages? 

Also to fix the below error please move all the file starting with cloudera from /etc/yum.repos.d to /tmp or any other location

# mv /etc/yum.repos.d/cloudera* /tmp

# yum clean all

# rm -rf /var/cache/yum

# yum repolist

BEGIN sudo yum --disablerepo=* --enablerepo=cloudera* info openjdk8
Error: Cannot find a valid baseurl for repo: cloudera-manager
END (1)
remote package openjdk8 is not available, giving up

[#4]

Hi @upadhyayk04,

Thanks for your answer.

• Firstly please confirm whether are you using the local repository for the installation of CDP. -- No, I'm not. I'm just following the steps to Installing a Trial Cluster
• to fix the below error please move all the file -- I've moved the files and error still persists.

BEGIN sudo yum --disablerepo=* --enablerepo=cloudera* clean all
74 files removed
END (0)
BEGIN sudo rm -Rf /var/cache/yum/*
END (0)
BEGIN sudo yum --disablerepo=* --enablerepo=cloudera* makecache
Error: Cannot find a valid baseurl for repo: cloudera-manager
END (1)
Installing openjdk8 package...
BEGIN sudo yum --disablerepo=* --enablerepo=cloudera* list installed openjdk8
Installed Packages
openjdk8.x86_64 8.0+232_9-cloudera @cloudera-manager
END (0)
BEGIN echo openjdk8 cloudera-manager-agent cloudera-manager-daemons | grep openjdk8
openjdk8 cloudera-manager-agent cloudera-manager-daemons
END (0)
BEGIN sudo yum --disablerepo=* --enablerepo=cloudera* info openjdk8
Error: Cannot find a valid baseurl for repo: cloudera-manager
END (1)
remote package openjdk8 is not available, giving up
waiting for rollback request

Here we have the result of executing the same commands in a terminal:

Here we have the content in the file cloudera-manager.repo

[cloudera-manager]
name=Cloudera Manager
baseurl=https://archive.cloudera.com/cm7/7.4.4/redhat8/yum/
gpgkey = https://archive.cloudera.com/cm7/7.4.4/redhat8/yum/RPM-GPG-KEY-cloudera
gpgcheck=1

Thank you in advice for any help!

[#5]

Hi @upadhyayk04 

Thanks for your answer.

• confirm whether are you using the local repository for the installation of CDP. -- No, I'm not. I'm just following the steps in the Installation Guide.
• to fix the below error please move all the file starting with cloudera. -- I have moved the file and error still persists, here we have the log.

Here we have the result of executing the same commands in a terminal:

Here we have the content of the file cloudera-manager.repo

What can be wrong?
Thank you in advice!

[#6]

Hello,

Thank you for your update

I could see enabled field is missing in Cloudera Manager repository

Can you please check the output of following command and see if packages are installed the yum is working fine

# rpm -qa | grep -i cloudera

# rpm -qa | grep -i jdk

# yum clean all

# yum repolist

[#7]

Thanks @upadhyayk04 

Here you have the output of those commands.

are they working fine?

[#8 · Accepted solution]

step1: Create a separate repository server, or alternatively, execute the following step at the location where you previously executed https://archive.cloudera.com/cm7/7.4.4/cloudera-manager-installer.bin

sudo yum install httpd -y sudo systemctl start httpd sudo systemctl enable httpd

sudo mkdir -p /var/www/html/cloudera-repos/cm7/

download as per your requirement redhat version redhat7 or redhat8 wget https://archive.cloudera.com/cm7/7.4.4/repo-as-tarball/cm7.4.4-redhat7.tar.gz OR wget https://archive.cloudera.com/cm7/7.4.4/repo-as-tarball/cm7.4.4-redhat8.tar.gz

sudo tar -xzvf cm7.4.4-redhat7.tar.gz -C /var/www/html/cloudera-repos/cm7/ OR sudo tar -xzvf cm7.4.4-redhat8.tar.gz -C /var/www/html/cloudera-repos/cm7/

sudo mkdir -p /var/www/html/cloudera-repos/cdh7/7.1.7.0/parcels

cd /var/www/html/cloudera-repos/cdh7/7.1.7.0/parcels

sudo wget https://archive.cloudera.com/cdh7/7.1.7/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976-el7.parcel sudo wget https://archive.cloudera.com/cdh7/7.1.7/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976-el7.parcel.sha1 sudo wget https://archive.cloudera.com/cdh7/7.1.7/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976-el7.parcel.sha256 sudo wget https://archive.cloudera.com/cdh7/7.1.7/parcels/KEYTRUSTEE_SERVER-7.1.7.0-1.keytrustee7.1.7.0.p0.159... sudo wget https://archive.cloudera.com/cdh7/7.1.7/parcels/KEYTRUSTEE_SERVER-7.1.7.0-1.keytrustee7.1.7.0.p0.159... sudo wget https://archive.cloudera.com/cdh7/7.1.7/parcels/KEYTRUSTEE_SERVER-7.1.7.0-1.keytrustee7.1.7.0.p0.159... sudo wget https://archive.cloudera.com/cdh7/7.1.7/parcels/manifest.json

To see on web browser http://<Repo_public_ip>/cloudera-repos/ Eg.: - http://13.233.244.230/cloudera-repos/

step 2

1	sudo vi /etc/yum.repos.d/cloudera-manager.repo
2	[cloudera-manager] name=Cloudera Manager baseurl=http://<Repo-server-private-ip>/cloudera-repos/cm7/cm7.4.4/ gpgkey =http:// <Repo-server-private-ip>/cloudera-repos/cm7/cm7.4.4/RPM-GPG-KEY-cloudera gpgcheck = 0

step 3

1	sudo yum clean all
2	sudo yum makecache
3	sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server -y

step 4

sudo systemctl start cloudera-scm-server

sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

step 5

click on parcel repositories & network settings

Then delete existing links , click on +

copy-paste following path http://Your_repo_private_IP/cloudera-repos/cdh7/7.1.7.0/parcels/

I hope these steps are helpful to you! 🙂 

[#9]

Hi @omkar_gaikwad 

I'm getting the following error when the command "sudo yum makecache" is executed in Step 3.

Cloudera Manager
Errors during downloading metadata for repository 'cloudera-manager':
- Status code: 502 for http://<IP>/cloudera-repos/cm7/cm7.4.4/repodata/repomd.xml (IP: xx.xx.xx.xx)
Error: Failed to download metadata for repo 'cloudera-manager': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

I executed the command with and without proxy configuration also with the proxy set in /etc/yum.conf file. 

Any idea what configuration is missing?

Thanks!!!

[#10]

Could you kindly confirm the following information for me?

1. Are you installing Cloudera using cloud services or on a local machine such as a laptop/PC?
2. What is the operating system of your system?

Your confirmation will help me to create a script with the necessary steps. I will share that script with you once it's ready. Thank you!