Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Who Agreed with this topic

Cloudera Mgr on AWS gives incorrect URL for Hue Web UI

Explorer

I have a small test cluster built in AWS running Cloudera manager 5.8.1-1 and CDH 5.8.0 w/parcels. Cloudera Manager is running on an instance in a private VPC. There is a 2 instance cluster in a public VPC which Cloudera Manager is managing using the internal FQDN for most things:

ip-10-x-x-x.us-west-1.compute.internal 

 

However, the Hue Web UI link in the CM Hue page gives the external IP address:

http://ec2-54-##-##-##.us-west-1.compute.amazonaws.com:8888.

 

I am not even sure where or how it is getting this value. It is internet routable FQDN for the instance, but there is no reference to in /etc/hosts or any other place I can find.

 

Even stranger, the hue file browser does not work for the same reason. When you click on the file browser link in the hue webui (after manually entering the correct internal IP address for the URL) it hangs and eventually this gets logged to /var/log/hue/error_log:

WebHdfsException: HTTPConnectionPool(host='ec2-54-##-##-###.us-west-1.compute.amazonaws.com', port=50070): Max retries exceeded with url: /webhdfs/v1/user/hive/warehouse?op=GETFILESTATUS&user.name=hue&doas=hdfs (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f5a18cf3310>: Failed to establish a new connection: [Errno 110] Connection timed out',)

 

I cannot figure out how to make Hue use the internal (10.x.x.x) address for the file browser. We do not want to open up port 50070 to the internet so we need to use the internal interface. In CM Hue configuration there is a radio button which allows you to toggle on/off the webhdfs URL:

HDFS Web Interface Role

webhdfs_url

 

In the value field for webhdfs_url it does not show the full name, but the portion it does show looks like it should be using the correct internal IP address (not the external one):

NameNode (ip-10-##-##-##)
Reset to empty default value

 

However, when I look at /var/run/cloudera-scm-agent/process/###-hue-HUE_SERVER I see that it is set to the external address:

webhdfs_url=http://ec2-54-##-##-##.us-west-1.compute.amazonaws.com:50070/webhdfs/v1

 

This is strange because all the other URLs in hue.ini use the correct internal address:

resourcemanager_api_url=http://ip-10-x-x-x.us-west-1.compute.internal:8088/
proxy_api_url=http://ip-10-x-x-x.us-west-1.compute.internal:8088/
history_server_api_url=http://ip-10-x-x-x.us-west-1.compute.internal:19888/
oozie_url=http://ip-10-x-x-x.us-west-1.compute.internal:11000/oozie

 

In CM Hue configuration I tried disabling the webhdfs URL radio button ("reset to default empty value") but it gives a validation error when I try to save the change. I also have set a safety valve for webhdfs URL in "Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini" but it does not override the webhfs_url which CM is setting in hue.ini:

hue_safety_valve.ini:webhdfs_url=http://ip-10-x-x-x.us-west-1.compute.internal:50070/webhdfs/v1
hue_safety_valve_server.ini:hadoop.hdfs_clusters.default.webhdfs_url=http://ip-10-x-x-x.us-west-1.compute.internal:50070/webhdfs/v1

 

I am wondering where CM is getting this external FQDN ec2-54-##-##-##.us-west-1.compute.amazonaws.com -- I find no reference to it on the server itself. I hope there is some way to manually set this value to the internal one. Any help would be appreciated.

 

Who Agreed with this topic