Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Access KNOX Admin UI from AMBARI (HDP 3.1)

Highlighted

Access KNOX Admin UI from AMBARI (HDP 3.1)

New Contributor

Hi

Created a cluster via cloudbreak from the blueprint. Trying to access Knox Admin UI from Ambari with no luck. URI points to https://<master_public_ip>:8443/<cluster_name>/manager/admin-ui, so I got 404. Investigated topologies and there is no topology named `manager`, only `admin.xml`, `default.xml`, `dp-proxy.xml` and `knoxsso.xml`. I'd like to secure Apache Livy Server with Knox. I Can successfully auth to YARNUIV2 or to WEBHDFS with curl, for example:

 

 

curl -iku admin:password -X GET 'https://<master_public_ip>:8443/<cluster_name>/dp-proxy/webhdfs/v1/?op=LISTSTATUS'

 

 

forks fine. How I could access Knox Admin UI or secure Livy Server with Knox?

 

I'm using:

Cloudbreak: 2.9.1

Ambari: 2.7.4.0

HPD: 3.1.4.0-315

Blueprint: HDP 3.1 - Data Science: Apache Spark 2, Apache Zeppelin

 
2 REPLIES 2

Re: Access KNOX Admin UI from AMBARI (HDP 3.1)

New Contributor

For internet searchers:

 

Livy can be accessed via 

https://<master_public_ip>:8443/<cluster_name>/dp-proxy/livy/v1, for example

 

curl -iku admin:password -X GET 'https://<master_public_ip>:8443/<cluster_name>/dp-proxy/livy/v1/sessions'

 

Username, password and cluster_name are set when cluster is created by cloudbreak. Master_public_ip is ip of ther VM which serves as ambari

 

Here is a python snippet to test it working:

 

import json, pprint, requests, textwrap
session_url = "https://<master_public_ip>:8443/<cluster_name>/dp-proxy/livy/v1/sessions"
headers = { 'X-Requested-By': 'livy'}
data = {'kind': 'spark'}
basic_auth = ('admin', 'password')
response = requests.post(session_url, headers=headers, auth=basic_auth, data=json.dumps(data), verify=False)
print(response.status_code)
print(response.text)
response.json()

 

Re: Access KNOX Admin UI from AMBARI (HDP 3.1)

New Contributor

I will answer to myself and for those who will search.

 

Start Demo LDAP from Ambari UI as manual said.

 

It looks like admin-ui doesn't enabled by default. So I created manager.xml in 

/etc/knox/conf/topologies/manager.xml with content (got it from knox github):
 

 

 

 

<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements.  See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License.  You may obtain a copy of the License at
    <a href="<a href="<a href="<a href="http://www.apache.org/licenses/LICENSE-2.0" target="_blank">http://www.apache.org/licenses/LICENSE-2.0</a>" target="_blank"><a href="http://www.apache.org/licenses/LICENSE-2.0</a" target="_blank">http://www.apache.org/licenses/LICENSE-2.0</a</a>>" target="_blank"><a href="<a href="http://www.apache.org/licenses/LICENSE-2.0</a" target="_blank">http://www.apache.org/licenses/LICENSE-2.0</a</a>" target="_blank"><a href="http://www.apache.org/licenses/LICENSE-2.0</a</a" target="_blank">http://www.apache.org/licenses/LICENSE-2.0</a</a</a>>>" target="_blank"><a href="<a href="<a href="http://www.apache.org/licenses/LICENSE-2.0</a" target="_blank">http://www.apache.org/licenses/LICENSE-2.0</a</a>" target="_blank"><a href="http://www.apache.org/licenses/LICENSE-2.0</a</a" target="_blank">http://www.apache.org/licenses/LICENSE-2.0</a</a</a>>" target="_blank"><a href="<a href="http://www.apache.org/licenses/LICENSE-2.0</a</a" target="_blank">http://www.apache.org/licenses/LICENSE-2.0</a</a</a>" target="_blank"><a href="http://www.apache.org/licenses/LICENSE-2.0</a</a</a" target="_blank">http://www.apache.org/licenses/LICENSE-2.0</a</a</a</a>>>>
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<topology>
<name>manager</name>
<gateway>
    <provider>
        <role>webappsec</role>
        <name>WebAppSec</name>
        <enabled>true</enabled>
        <param>
            <name>csrf.enabled</name>
            <value>true</value>
        </param>
        <param>
            <name>csrf.customHeader</name>
            <value>X-XSRF-Header</value>
        </param>
        <param>
            <name>csrf.methodsToIgnore</name>
            <value>GET,OPTIONS,HEAD</value>
        </param>
        <param>
            <name>xframe.options.enabled</name>
            <value>true</value>
        </param>
        <param>
            <name>xss.protection.enabled</name>
            <value>true</value>
        </param>
        <param>
            <name>strict.transport.enabled</name>
            <value>false</value>
        </param>
    </provider>
    <provider>
        <role>federation</role>
        <name>SSOCookieProvider</name>
        <enabled>true</enabled>
    </provider>
    <provider>
        <role>identity-assertion</role>
        <name>HadoopGroupProvider</name>
        <enabled>true</enabled>
        <param>
            <name>CENTRAL_GROUP_CONFIG_PREFIX</name>
            <value>gateway.group.config.</value>
        </param>
    </provider>
    <provider>
        <role>authorization</role>
        <name>AclsAuthz</name>
        <enabled>true</enabled>
        <param>
            <name>knox.acl.mode</name>
            <value>OR</value>
        </param>
        <param>
            <name>knox.acl</name>
            <value>KNOX_ADMIN_USERS;KNOX_ADMIN_GROUPS;*</value>
        </param>
    </provider>
</gateway>
<service>
    <role>KNOX</role>
</service>
<application>
    <name>admin-ui</name>
</application>
</topology>

 

 

 

Don't forget to change the owner: 

 

 

 

chown knox:knox /etc/knox/conf/topologies/manager.xml

 

 

 

Check existing of /etc/knox/conf/topologies/knoxsso.xml
Check `redirectToUrl` in `Advanced knoxsso-topology` in Ambrai Knox Configs. It should contain cluster name (mine was configured to /default/)
Check topologies for LDAP configuration, they should be configured to query `ou=people,dc=hadoop,dc=apache,dc=org` (Demo LDAP), in my case some of them were configured to `ou=Users,dc=hadoop,dc=apache,dc=org`
Restart ldap
Restart Knox
Test:

 

 

 

curl -iku admin:admin-password -X GET 'https://<master_public_ip>:8443/<clustername>/dp-proxy/livy/v1/sessions'

 

 

 

Test admin UI:

 

 

 

https://<master_public_ip>:8443/<clustername>/manager/admin-ui/

 

 

 

 

PS. Somehow if I query LDAP with ldapseach at master node and localhost:33389 it returns me users with randomly generated passwords. But users.ldif file has simple passwords as it configured in Ambari. And simple passwords work, so what's wrong with ldap at localhost:33389 and which LDAP queried by Knox?

 
Don't have an account?
Coming from Hortonworks? Activate your account here