Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Problem with AWS Cloudera Quick Start Deployment Step 3 Connect to Director on port 7189

avatar
Contributor

Hello

I  am using the AWS quick start deployment guide here

 

http://docs.aws.amazon.com/quickstart/latest/cloudera/step3.html

 

 

I am using the Quick Start guide because I am new to this subject so thought this would be the best way to go and expected that the guide is Newbie/Idiot proof.

 

I have created 2 RHEL-7.1 nodes, and it all goes well. However I am stuck at step 3 trying to connect to the Cloudera Director. When I enter the IP and port address into my browser (tried Safari and Chrome) no connection is made and the browsers time out the connection. I checked a lot of similar posts and tried lots of things but no luck. Its not clear to me if

 

1. The IP address I use in the SSH command in step 3.2 is the public or private IP address for my Cluster Launcher Node?

2. Do I enter this ssh tunnel command in a new terminal window on my machine (MacBook running 10.10.4) or the terminal window I used in Step 3.1 ?

2. Some posts reference port 7180, what IS the correct port 7180 or 7189?

3. Is There anything I need to do in my browser? I already followed the instructions to add the ProxySwitch Omega to Chrome and that did not work .

4. I have Cloudera Director running as seen in the ps - ef command, am I missing anything? Are there any steps missing from the guide? I am assuming lots of things you would normally do in a manual install are done 'behind the scenes' by the AWS CloudFormation template, but I suspect its missing something.

 

Been stuck on this for weeks now, like many other people I have seen on various posts, so any new suggestions greatly appreciated

 

Thanks.

 

 

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hi Cerno,

 

With the SSH tunnel established to port 7189 of your Director instance, try instead connecting to http://localhost:7189/. That connects you to the local end of the tunnel, which on the other end will talk to the Director process.

 

If that does not work, use netstat on the Director instance to verify that the Director process is listening on port 7189. If it is, then do a curl or wget on http://localhost:7189 on the Director instance itself to make sure that Director is responding to any requests.

 

Hopefully the above steps will either get you connected or narrow down where the problem lies.

 

Bill

 

P.S. Just to be clear, when you use the SSH tunnel for port 7189, the security group(s) for the Director instance do not need to allow any outside access to port 7189. They only need to allow access to port 22, so that the tunnel can reach through over SSH. Since you're able to SSH in, that seems to be working.

View solution in original post

4 REPLIES 4

avatar
Expert Contributor

Cerno,

 

Here's some answers to your questions.

 

1. The public IP or hostname should be used. The quickstart creates a new VPC and you will not be able to resolve private IPs from outside the VPC.

 

2. You enter the tunnel command in a new terminal window on your machine. I looked at the instructions and I did not see a terminal window created in Step 3.1. Step 3.1 looks like it's for determining what the ssh command is. I.e., Step 3.1 shows that you use the public IP to ssh along with the user name since some amis use different user names (e.g., "ec2-user", "root", or "centos" are common usernames). The username for the ClusterLauncher is  "ec2-user".

 

2. Cloudera Director runs on port 7189. Cloudera Manager runs on port 7180. You will be tunnelling to 7189 for access to Cloudera Director.

 

3. ProxySwitch Omega is used in conjunction with a SOCKS proxy to allow you to access all cluster instance UIs through your browser without the need to tunnel each connection individually. You need to activate the proxy profile in your browser as well open an ssh connection with the -CD <port> option. Can you describe the steps taken and your symptoms in more detail? We may be able to help if you provide some more information on the problems you are encountering.

 

4. The quickstart installs Cloudera Director from RPM and then seeds it with an environment and instance templates for your convenience. If you are manually installing Cloudera Director in your own VPC you would need to manually create the Environment and InstanceTemplates to point to the correct subnets, security groups, amis, credentials, etc. I think everything is ready if you have confirmed that Cloudera Director is running.

 

Also note that AWS has a forum for discussing quick start issues. You may find answers to issue with the Cloudformation template there.

https://forums.aws.amazon.com/forum.jspa?forumID=178

 

David

avatar
Contributor

Hi David

 

Thank you for your quick reply. 

 

1. I checked that director is running here is the output.

 

clouder+ 509 506 0 06:30 ? 00:00:00 bash /usr/lib64/cloudera-director/server/bin/cloudera-director-server
clouder+ 551 509 0 06:30 ? 00:01:42 /usr/lib/jvm/jre-openjdk/bin/java -XX:MaxPermSize=256M -XX:+CMSClassUnloadingEnabled -Xms512m -Xmx1g -Dpid.file=/var/run/cloudera-director-server/application.pid -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Dlogging.config=/etc/cloudera-director-server/logback.xml -Dloader.path=lib/ -cp /usr/lib64/cloudera-director/server/lib/launchpad-server-2.0.0.jar org.springframework.boot.loader.PropertiesLauncher --spring.config.location=/etc/cloudera-director-server/application.properties --lp.access.logging.config.file=/etc/cloudera-director-server/logback-access.xml --lp.database.path=/var/lib/cloudera-director-server/state
ec2-user 3169 3145 0 11:47 pts/1 00:00:00 grep --color=auto director
[ec2-user@ip-10-0-2-12 ~]$

 

2. I tunneled to the public IP address in another window

ssh -i "Fincerno.pem" -L 7189:localhost:7189 ec2-user@ec2-52-28-166-163.eu-central-1.compute.amazonaws.com
Last login: Fri Jul 8 11:41:06 2016 from 90.201.159.230
[ec2-user@ip-10-0-2-12 ~]$

 

3. Then tried Safari to conenct to the public IP on port 7189

http://52.28.166.163:7189 but safari hangs and says the server unexpectedly dropped the connection

 

4. Tried the same from Chrome and it says the address cant be reached.

 

5. I checked in the security groups and all IPs and port ranges seem to be open.

 

Not sure where to go from here?

Regards

David.

 

 

 

 

avatar
Super Collaborator

Hi Cerno,

 

With the SSH tunnel established to port 7189 of your Director instance, try instead connecting to http://localhost:7189/. That connects you to the local end of the tunnel, which on the other end will talk to the Director process.

 

If that does not work, use netstat on the Director instance to verify that the Director process is listening on port 7189. If it is, then do a curl or wget on http://localhost:7189 on the Director instance itself to make sure that Director is responding to any requests.

 

Hopefully the above steps will either get you connected or narrow down where the problem lies.

 

Bill

 

P.S. Just to be clear, when you use the SSH tunnel for port 7189, the security group(s) for the Director instance do not need to allow any outside access to port 7189. They only need to allow access to port 22, so that the tunnel can reach through over SSH. Since you're able to SSH in, that seems to be working.

avatar
Contributor

Hi Bill,

 

Using localhost worked! Just used Safari to connect.

 

Many Thanks

 

David