Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Cloudera Manager Hangs When Installing Parcels

avatar
Explorer

Hello,

Been working with Cloudera for about 2 years now and up until recently, when the Cloudera Express 5.9 version was released, I had no issue adding a cluster. Starting with 5.9, when adding a cluster, I can find and select the hosts, parcels, install the CM agent and get a good status on a heartbeat. When the Installing Selected Parcels step comes in, I show 100% downloaded, but 0/0 in Distributed, Unpacked, and Activated. The issue exists regardless of which parcel or how many types I choose to install. My logs show the new cluster has no member hosts. I then have to open a new window, add the hosts to the cluster, distribute and upgrade CDH to get the parcels to distribute, unpack, and activate.

 

I thought it may have been a bug within the Cloudera Express 5.9.1, so I stood a new server up and installed the trial version of Enterprise Data Hub 5.10.1 only to have the same result. I can't find Cloudera reporting this as a known issue or any other instances of this exact problem. Can someone please help?

 

My configuration is an AWS environment with a single RHEL 6.8 (CE 5.9.1) and RHEL 7.3 (EDH 5.10.1) server; I am not running single-user mode.

 

Below is from the Enterprise Data Hub trial version 5.10.1, however the same errors and issues exist in the Express 5.9.1 version.

 

 

An excerpt of the log:

 

2017-03-30 16:13:48,639 WARN ParcelUpdateService:com.cloudera.parcel.components.ParcelInstallerImpl: Error while installing parcel
com.cloudera.parcel.ParcelException: Cluster <b>cluster</b> has no member hosts. Add hosts to the cluster and then try again.
        at com.cloudera.parcel.components.ParcelManagerImpl.distribute(ParcelManagerImpl.java:362)
        at com.cloudera.parcel.components.ParcelInstallerImpl.handleJob(ParcelInstallerImpl.java:173)
        at com.cloudera.parcel.components.ParcelInstallerImpl.run(ParcelInstallerImpl.java:145)
        at com.cloudera.parcel.components.ParcelInstallerImpl.run(ParcelInstallerImpl.java:34)
        at com.cloudera.cmf.persist.ReadWriteDatabaseTaskCallable.call(ReadWriteDatabaseTaskCallable.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

 

 

Screenshot of where it hangs:

clusterinstallhang.JPG

 

Back at the Dashboard

Cluster-no-hosts.JPG

The hosts have the agent installed and appear in good condition:

cluster-allhosts.JPG

 

Thanks in advance.

 

1 ACCEPTED SOLUTION

avatar
Explorer

Thank you for working with me via messages to resolve my issue. 

 

As you stated in your message to me, FQDN resolution was the cause of my problem. Since we're not using a DNS solution for our environment, once I entered the cluster hosts FQDNs into the CM's /etc/hosts, the parcels no longer hung at distribution and the installation was able to continue.

View solution in original post

9 REPLIES 9

avatar
Expert Contributor

Hello,

 

Thank you for reaching out to us, in order to assist you we would need to see additional log information from Cloudera Manager. The error you have referenced indicates that no host have been added to the cluster you are attempting to install parcels to. Based on what is here at this moment this does not appear to be a bug or a change in our software behavior.

 

In addition to that the error you provided has a differeing cluster name than the ones you provided in your screenshots. We also note that your screenshots show 'Cluster 1' as having CDH 5.0.0 deployed via packages and not via parcels. The All Host page does not show us which host are presently assigned to a specific cluster without filtering.

 

If you have added host to Cluster 1 or another cluster please use the 'Cluster' filter on the side of the All Host page so that we can see which host are assigned to that cluster. Please confirm the cluster name, CDH release, Packaging method, and any previous deployment steps you have taken.

---
Customer Operations Engineer | Security SME | Cloudera, Inc.

avatar
Explorer

Thank you for working with me via messages to resolve my issue. 

 

As you stated in your message to me, FQDN resolution was the cause of my problem. Since we're not using a DNS solution for our environment, once I entered the cluster hosts FQDNs into the CM's /etc/hosts, the parcels no longer hung at distribution and the installation was able to continue.

avatar
New Contributor

Hi Me too facing the same issue today. how to enter FQDNs into the CM's /etc/hosts. Kindly guide me

 

 

avatar
New Contributor

I am doing cloudera manager instalation in AWS ec2 cloud. 3 hosts are there in my cluster. I think this is the last step to get my complely installed cloudera manager.

avatar
New Contributor

Is there any experts to chat and solve my isssue

avatar
New Contributor

Hello baghdaddy,

 

How have you fixed this problem? Can you explain it to me? I'm facing the same problem.

avatar
New Contributor

I could fix this issue while installing CDP Data Center Trial.

First, you need to get hostname of all hosts in your cluster.

After that, try to edit /etc/hosts file as below:

<host 1's local IP> <hostname1>

<host 2's local IP> <hostname2>

Save it and restart hosts.

avatar
New Contributor

Adding to this response, my issue was I missed to insert the CM's hostname+ip in the /etc/hosts config file of the client nodes.

While the installation was still stuck I added the missing line to the /etc/hosts file on the client machine and the installation moved on fine.

 

 

 

avatar
New Contributor

Another edge case that you might hit is if a host has both IPv6 and IPv4 address (and only really have IPv4 connectivity due to firewalls etc).  Running "lsof -p (pid-of-flood-process)" on the agent box and seeing "SYN_SENT" was the clue here.