Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Recommendation for proxying Hadoop services without built-in Knox support

avatar

Knox 0.6.0 has built-in support for these 7 services:

  • WebHDFS
  • WebHCat
  • Oozie
  • HBase
  • Hive
  • Yarn
  • Storm

Is there a recommended approach to expose other services from the gateway host? Particularly web UIs, such as Ambari & Ranger.

1 ACCEPTED SOLUTION

avatar

These extensions are committed to the Apache Knox repo itself. They all use the config driven extension model so you need to look in the gateway-service-definitions module. In particular look in this directory. Now that you mention the openweathermap example, I need to update that to the new configuration based model at least as a comparison to the code based extension. The developers guide does briefly cover the config based extension.

View solution in original post

13 REPLIES 13

avatar
Master Mentor

avatar
Master Guru

I suppose you can use haproxy for example. However if you have kerberos and spnego you would need to add the proxy tickets similar to the oozie ha setup described here in the cloudera doc ( I would use ours if we would actually describe that ) http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cdh_sg_oozie_ha_ker...

avatar

I figured something like haproxy or nginx would work. Preferably looking for an example config, or if anyone has extended Knox with a custom provider then even better.

avatar

We would certainly recommend the use of Knox's extensibility models to cover any components without coverage before we get there ourselves. There have been several developed in the community already such as Falcon that we don't yet officially support. The same goes for UI coverage where the community has added coverage for things like the HDFS and YARN UIs among others. The Knox Developer's Guide is a great resource that the community has used to help them jump start these efforts. Of course looking at the implementation of the existing integrations is a great place to start as well.

avatar

The openweathermap example in the Knox Dev Guide looks great as a reference for extending Knox yourself. Do you know where some existing community extensions, like the Falcon or NN/RM UIs, can be found? I checked the Hortonworks Gallery with no luck.

avatar

These extensions are committed to the Apache Knox repo itself. They all use the config driven extension model so you need to look in the gateway-service-definitions module. In particular look in this directory. Now that you mention the openweathermap example, I need to update that to the new configuration based model at least as a comparison to the code based extension. The developers guide does briefly cover the config based extension.

avatar

avatar
Master Mentor

@Kevin Minder moving to the best answer.

avatar

For quick reference, here's an example of adding Oozie UI to HDP 2.4 Sandbox:

1. start Sandbox and make sure all non-maintenance services are running

2. add service definition:

git clone https://git-wip-us.apache.org/repos/asf/knox.git
cp -R knox/gateway-service-definitions/src/main/resources/services/oozieui /var/lib/knox/data-2.4.0.0-169/services/
chown -R knox:knox /var/lib/knox/data-2.4.0.0-169/services/oozieui

3. add OOZIEUI service to default.xml topology (Ambari > Knox > Configs > Advanced topology)

<service>
    <role>OOZIEUI</role>
    <url>http://{{oozie_server_host}}:{{oozie_server_port}}/oozie</url>
  </service>

4. start (or restart) Knox & Demo LDAP (using Ambari)

5. visit https://localhost:8443/gateway/default/oozie/