CLARIFICATION:   When I said:   "Setting http_host will not impact this issue as Cloudera Manager sets its own host to use for Hue monitoring."   That was a bit out of context.  I meant that if you set "http_host" in the Hue config [desktop] section, it will not impact the "http_host" chosen by Cloudera Manager if Bind Hue Server to Wildcard Address is enabled in the Hue configuration.   -Ben ... View more

@jehalter   Indeed, the cloudera-monitor.properties includes the configuration.  Hadn't considered it before you brought that up, but since the agent pulls from that file for its configuration, after you have started Hue, you can edit that file and set the "http_host" to what you want.  A restart of the Cloudera Manager agent on the Hue host would pick up your changes when it initializes the monitor. Setting http_host will not impact this issue as Cloudera Manager sets its own host to use for Hue monitoring.   I've opened a Jira internally here with the CM team to assess how to fix this long-term.   For now, we need to make sure the "allowed_hosts" workaround is working or, if you are OK with disabling the Bind Hue Server to Wildcard Address setting so that Hue listens on a specifc hostname, then we can just go with that as another workaround.   I consistently have good luck with the allowed_hosts=* fixing this and have also adopted the following to retain some security:   [desktop] allowed_hosts=.cloudera.com,0.0.0.0,192.168.114.79   The above allows access from any host in the "cloudera.com" domain who is connecting to either 0.0.0.0 or 192.168.114.79   Alternatively, you could do the following:   (1)   Start Hue (observe the bad health)   (2)   cd to the following directory: /var/run/cloudera-scm-agent/process/`ls -lrt /var/run/cloudera-scm-agent/process/ | awk '{print $9}' |grep HUE_SERVER| tail -1`   (3)   Edit cloudera-monitor.properties   change "http_host" value to the value returned by "hostname -f"   Save   (4)   Restar the agent on that host with:   service cloudera-scm-agent restart   This kind of thing won't last long, though, since the cloudera-monitor.properties will be recreated when you restart Hue again.  Probaby not the best solution in that case.   Ben ... View more

@jehalter,   When you posted your screen shots, that had to have been after you unchecked Bind Hue Server to Wildcard Address as Hue would have then showed http_host as "0.0.0.0".   The only oddity in your case seems to be that allowed_hosts=* is not working for you.  I belive it was working, but the agent monitor thread had already cached the http_host so the agent continued to use 0.0.0.0.   I would recommend (to you, too, @SRG) trying the following with Bind Hue Server to Wildcard Address set to true:   (1)   Edit Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini in Cloudera Manager   Make sure the desktop section has:   [desktop] allowed_hosts=*   (2)   Save the configuration change   (3)   Restart Hue from Cloudera Manager   (4)   Restart the agent on the host where Hue runs with:   service cloudera-scm-agent restart   (5)   Let us know if that allows the Hue host check to succeed.  If it fails,  please see if you can find the stack/error in the /var/log/cloudera-scm-agent/cloudera-scm-agent.log file.   If that works for all of us, then we can maybe tweak the allowed_hosts value to be more secure but still let the agent do its health check.   Thank you,   Ben     ... View more

@jehalter and @mbigelow,   I looked at how http_host was set in Cloudera Manager for Hue and found that if Bind Hue Server to Wildcard Address is set in the Cloudera Manager Hue configuration, then the monitor will use the wildcard (0.0.0.0).  I backed out my hack of the agent code and can now get the same behavior.   Still, my agent has no trouble talking to Hue even with "Bind Hue Server to Wildcard Address" set in Cloudera Manager as long as I have "allowed_hosts=*" set.   I feel like there is a bug here... I'll sort it out and make sure we address it in the long term.   Let me know if any of this helps and if you can get screen shots of your Hue configuration.   Thanks,   Ben ... View more

I tested by forcing the agent code to use "0.0.0.0" as the http_host for the Hue health check. This generated the exception you saw when the check was done:   [08/Feb/2017 09:27:22 +0000] 7376 Monitor-GenericMonitor throttling_logger ERROR Error calling is alive at 'http://0.0.0.0:8888/desktop/debug/is_alive' Traceback (most recent call last): File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/monitor/generic/hue_adapters.py", line 39, in collect_and_parse self._call_is_alive(self._metrics_uri) File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/monitor/generic/hue_adapters.py", line 76, in _call_is_alive head_request_with_timeout(is_alive_url, timeout=timeout) File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/url_util.py", line 94, in head_request_with_timeout max_cert_depth) File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/url_util.py", line 67, in urlopen_with_timeout return opener.open(url, data, timeout) File "/usr/lib64/python2.7/urllib2.py", line 437, in open response = meth(req, response) File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib64/python2.7/urllib2.py", line 475, in error return self._call_chain(*args) File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/https.py", line 205, in http_error_default raise e HTTPError: HTTP Error 400: BAD REQUEST   I then added the following to the following to my Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini, restarted Hue from Cloudera Manager, and the problem was resolved... the health check ran fine:   [desktop] allowed_hosts=*   My assessment, then, is that either Hue was not restarted from Cloudera Manager or the restart failed.  "allowed_hosts=*" will allow any clients to connect. If anyone has other ideas why my test results differ (with the allowed_hosts configuration change) please let us know.   Actually, in your Hue UI, please verify the "allowed_hosts" configuration that Hue is using in memory.  That will tell us.   - Log into Hue as a Hue superuser. - Click on the "Configuration" subtab - Under "Configuration Sections and Variables" click "desktop" - Search with your browser search for "http_host" (record the values and share with us) - Search with your browser for "allowed_hosts" (record the values and share with us) Screen shots would be great.  Here are screen shots of mine:               ... View more

@jehalter and @mbigelow,   This gets more interesting. The URL used in the hue health check is generated based on configurations returned by the Cloudera Manager heartbeat response.  I tried overriding the "http_host" config in the Hue safety valve to set it to 0.0.0.0 but that did not alter my URL at all. I checked and indeed the heartbeat response does not include the value for http_host.  I think this is a Cloudera Manager bug/issue, but I'll need to take a closer look.   That said, I decided to hack the hue_adapters.py code to just use the host string "0.0.0.0" and reproduced the issue.   The question I have then, is how is Cloudera Manager reporting "0.0.0.0".  That indicates to me that there is some problem with the hostname of the host as it appears in your Cloudera Manager Hosts tab.  Cloudera Manager will attempt to use the hostname that is reported by the agent to set Hue's http_host value to the hostname on which Hue is running.  If it is not able to do so, it is possible it leaves the Hue default which is "0.0.0.0"   I'll keep looking, but please check what your Hue host is named in Cloudera Manager's Hosts tab.   Thanks!   Ben ... View more

@jehalter,   Thank you for the information and I'm sorry to hear that it didn't help. I did understand that you were not having trouble with the browser; the agent acts as a client of Hue, so I was considering it likely that the new allowed_hosts feature was blocking the agent from connecting to Hue.   Just to be sure, you added the following to the [desktop] section in the safety valve:   allowed_hosts=*   If you don't add the [desktop] section header before it, then the configuration will be ignored.  As long as "allowed_hosts=*" is in the [desktop] section and you restarted, I'd assess that to be a clean test.   I'll let you know what we might try next.   Ben ... View more

@jehalter,   Actually, are you on SLES... this may be a bug   Thanks,   Ben ... View more

Hello @jehalter,   Actually, it is an interesting one.  The problem here is that in CDH 5.10, by default, Hue will restrict access to hosts based on the value of "socket.getfqdn()"   Based on the URL that is being used to try to contact Hue, it seems you have some hostname/dns issues here:   Error calling is alive at 'https://0.0.0.0:8888/desktop/debug/is_alive'   In any case, apart from maybe taking a look at how your host and domain are configured on the host, it appears that Hue is blocking the connection because the client's domain does not match allowed_hosts.   In short, I would recommend trying to run the following:     # python -c 'import socket;print socket.getfqdn()'   Then, take the result and add it to your current list of allowed_hosts. To do so:   (1)   In Cloudera Manager, navigate to Hue --> Configuration and then search for "Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini"   (2)   In the Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini text field, add the following:   [desktop] allowed_hosts=example.com   (replace "example.com" with the domain of your host)   If preferred and you just want to get this working, you can open up connections from all hosts by adding this instead:   [desktop] allowed_hosts=*   (3)   Restart Hue   CAUSE:   In CDH 5.10, security was tightened in Hue by only allowing clients that are in the same domain as Hue to connect to Hue.  It appears your hostname is actually 0.0.0.0 on the host, so the real fix would be to ensure that the host is configued to have a valid hostname and FQDN.   Let us know if you have questions about the steps or they don't work.   Ben ... View more