cloudera deployment on aws - deploy client configuration fails

[#1]

Hello,

I'm trying to install a cloudera distibution on AWS.

In my current installation I was able to install Cloudera manager on AWS.

I have specified three hosts which will be used for my cluster.

The moment Cloudera manager wants to deploy the client configurtation to the hosts, the config keeps on hanging.

I can acces the hosts by ssh and I can acces cloudera manager. For some reason the clouders manager is not able to communicate with its hosts. Is there a manual config I need to do ?

Any idea how I can go around this ?

Any hulp much appreciated because it's a showstopper.

[#2]

I think you aren't setting up the /etc/hosts file properly. You'll need to have it configured with all of the hosts for your entire cluster. When you go to add hosts, use the host names from /etc/hosts instead of the AWS ones.

If this is a POC, I'd recommend using the built-in AWS spin up wizard. It takes care of the names for you.

[#3]

When running the wizard for installing the cloudera hosts all goes fine until the moment clooudera manager wants to inspect the hosts.

So I used the wizard but still there are issues.

I have tried already to edit the etc/hosts and put the interal address in the hosts and then I tried to rerun the installation but the problem persists.

[#4]

I just got around this problem.

The problem is that cloudera manager is trying to contact the host by using port 9000.

So I added port 9000 to my security group config.

Now I'm at the point where the client config will be deployed. Let's hope this goes fine.

Maybe I need to identify all the ports being used for the deployment on AWS.

[#5]

Hi Koen

Our blog post at http://blog.cloudera.com/blog/2013/03/how-to-create-a-cdh-cluster-on-amazon-ec2-via-cloudera-manager... gives a good starting list of ports you'll want to ensure are in your security group config. See also http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Instal... for a full list of possible ports for inter/intra-node communication.

Regards,

--

Mark

[#6 · Accepted solution]

Hey.

Having the list of all the ports being used helped me a lot in the next problem I faced.

After the deployment went fine I went back to the cloudera manager screen and the main page was not showing the graphs and statistics. There was a message saying time-out to the host.

So I added the list of all the Cloudera ports to the security group and now the Cloudera manager is showing the graphs.

So adding the ports into the secuirty group fixed the issue of not seeing the graphs in Cloudera manager.

Any chance you know which port is responsible for graph communication ?

Thanks a lot guys, Kudos !