Support Questions

ask_bill_brooks · ‎10-21-2019

I have had a running 5 node (1 Master) cloudera cluster for the past 1 week and all of a sudden I have lost WEB UI access to Cloudera Manager.

I am able to work on other services like HIVE/HBASE/SPARK/NIFI etc. The status shows that both the cloudera agent and cloudera server are running. I am able to run the Curl command on 7180 on the Master node and I get the output.

Not able to figure out what the problem could be.

Axe · ‎10-22-2019

There is no firewall installed on the EC2 instance and the security group rules (that are common for all nodes) are configured to allow all traffic from my IP.

Fei Yu · ‎10-22-2019

Hi Axe,

You mentioned that ". I am able to run the Curl command on 7180 on the Master node and I get the output. "

When you said you get the output, do you mean the curl response content is the correct CM web UI content? If yes, it means CM server itself works fine with the Web UI part. Can you try to curl to Master node 7180 from a different machine, including the machine you are trying to connect using web browser and see if you get the correct response.

Axe · ‎10-22-2019

Curl Response from Cloudera Manager Host.

[root@ip-172-31-24-21 etc]# curl -u admin:admin http://ec2-18-144-47-252.us-west-1.compute.amazonaws.com:7180 -v

* About to connect() to ec2-18-144-47-252.us-west-1.compute.amazonaws.com port 7180 (#0)

* Trying 172.31.24.21...

* Connected to ec2-18-144-47-252.us-west-1.compute.amazonaws.com (172.31.24.21) port 7180 (#0)

* Server auth using Basic with user 'admin'

> GET / HTTP/1.1

> Authorization: Basic YWRtaW46YWRtaW4=

> User-Agent: curl/7.29.0

> Host: ec2-18-144-47-252.us-west-1.compute.amazonaws.com:7180

> Accept: */*

>

< HTTP/1.1 200 OK

< Date: Tue, 22 Oct 2019 21:55:52 GMT

< Set-Cookie: CLOUDERA_MANAGER_SESSIONID=node0124m6i5b2ijzlyafpyc8wm6u724.node0;Path=/;HttpOnly

< Expires: Thu, 01 Jan 1970 00:00:00 GMT

< Last-Modified: Fri, 19 Jul 2019 06:26:10 GMT

< Content-Type: text/html;charset=utf-8

< Accept-Ranges: bytes

< Cache-Control: max-age=3600,public

< X-XSS-Protection: 1; mode=block

< X-Frame-Options: SAMEORIGIN

< X-Content-Type-Options: nosniff

< Content-Length: 63

<

* Connection #0 to host ec2-18-144-47-252.us-west-1.compute.amazonaws.com left intact

Curl Response from some other machine.

* Rebuilt URL to: http://ec2-18-144-47-252.us-west-1.compute.amazonaws.com:7180/

* Trying 18.144.47.252...

* TCP_NODELAY set

* Connection failed

* connect to 18.144.47.252 port 7180 failed: Operation timed out

* Failed to connect to ec2-18-144-47-252.us-west-1.compute.amazonaws.com port 7180: Operation timed out

* Closing connection 0

curl: (7) Failed to connect to ec2-18-144-47-252.us-west-1.compute.amazonaws.com port 7180: Operation timed out

I have all the necessary ports open on the CM Host machine for HTTP/HTTPS and also the 7180 port.

Fei Yu · ‎10-22-2019

It looks like a network relevant configuration issue. Can you please run below command on CM server host:

netstat -lnpt | grep 7180

Axe · ‎10-22-2019

Find the output below.

[root@ip-172-31-24-21 etc]# netstat -lnpt | grep 7180

tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 1250/java

Fei Yu · ‎10-22-2019

Are you even able to ping this host from your local machine? Are you able to telnet to other port from your local machine?

Axe · ‎10-22-2019

I am able to ping the server but not telnet it.

But I am able to connect to the server using scp and telnet is working on port 22.

Fei Yu · ‎10-22-2019

Hi Axe,

As you are able to curl port 7180 from the CM server host successfully, but not from other machines, this proves that CM server port 7180 is listening and working fine. You need to work with Network team or AWS team to investigate further regarding network relevant issue I believe. for example iptables or firewalld configuration. Please let us know if you have any progress.

Fei Yu · ‎10-22-2019

Another thing worth trying is to capture network traffic on CM server host for port 7180 (you can use tcpdump to do this for example), and check when you request to access port 7180 from your web browser, does CM server host even receive the request or not.

ssulav · ‎10-22-2019

I would suggest you to go through the below docs and verify the outbound rules on port 7180.

https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html

Cloudera Community

Support Questions

Not able to connect to Cloudera Manager Web UI