Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Recommendation for proxying Hadoop services without built-in Knox support

Solved Go to solution
Highlighted

Recommendation for proxying Hadoop services without built-in Knox support

Knox 0.6.0 has built-in support for these 7 services:

  • WebHDFS
  • WebHCat
  • Oozie
  • HBase
  • Hive
  • Yarn
  • Storm

Is there a recommended approach to expose other services from the gateway host? Particularly web UIs, such as Ambari & Ranger.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

These extensions are committed to the Apache Knox repo itself. They all use the config driven extension model so you need to look in the gateway-service-definitions module. In particular look in this directory. Now that you mention the openweathermap example, I need to update that to the new configuration based model at least as a comparison to the code based extension. The developers guide does briefly cover the config based extension.

View solution in original post

13 REPLIES 13
Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

I suppose you can use haproxy for example. However if you have kerberos and spnego you would need to add the proxy tickets similar to the oozie ha setup described here in the cloudera doc ( I would use ours if we would actually describe that ) http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cdh_sg_oozie_ha_ker...

Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

I figured something like haproxy or nginx would work. Preferably looking for an example config, or if anyone has extended Knox with a custom provider then even better.

Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

We would certainly recommend the use of Knox's extensibility models to cover any components without coverage before we get there ourselves. There have been several developed in the community already such as Falcon that we don't yet officially support. The same goes for UI coverage where the community has added coverage for things like the HDFS and YARN UIs among others. The Knox Developer's Guide is a great resource that the community has used to help them jump start these efforts. Of course looking at the implementation of the existing integrations is a great place to start as well.

Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

The openweathermap example in the Knox Dev Guide looks great as a reference for extending Knox yourself. Do you know where some existing community extensions, like the Falcon or NN/RM UIs, can be found? I checked the Hortonworks Gallery with no luck.

Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

These extensions are committed to the Apache Knox repo itself. They all use the config driven extension model so you need to look in the gateway-service-definitions module. In particular look in this directory. Now that you mention the openweathermap example, I need to update that to the new configuration based model at least as a comparison to the code based extension. The developers guide does briefly cover the config based extension.

View solution in original post

Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

Highlighted

Re: Recommendation for proxying Hadoop services without built-in Knox support

@Kevin Minder moving to the best answer.

Re: Recommendation for proxying Hadoop services without built-in Knox support

For quick reference, here's an example of adding Oozie UI to HDP 2.4 Sandbox:

1. start Sandbox and make sure all non-maintenance services are running

2. add service definition:

git clone https://git-wip-us.apache.org/repos/asf/knox.git
cp -R knox/gateway-service-definitions/src/main/resources/services/oozieui /var/lib/knox/data-2.4.0.0-169/services/
chown -R knox:knox /var/lib/knox/data-2.4.0.0-169/services/oozieui

3. add OOZIEUI service to default.xml topology (Ambari > Knox > Configs > Advanced topology)

<service>
    <role>OOZIEUI</role>
    <url>http://{{oozie_server_host}}:{{oozie_server_port}}/oozie</url>
  </service>

4. start (or restart) Knox & Demo LDAP (using Ambari)

5. visit https://localhost:8443/gateway/default/oozie/

Don't have an account?
Coming from Hortonworks? Activate your account here