Knox Response status: 401

[#1]

Hi all

I am trying figure out knox gateway

but I have problem when I access services like WEBHDFS

this is error log from /var/log/knox/gateway-audit.log:

17/12/28 21:30:30 ||de5c4e70-c89c-487e-8fea-6260c6701efb|audit|IPADDR|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1|unavailable|Request method: GET
17/12/28 21:30:30 ||de5c4e70-c89c-487e-8fea-6260c6701efb|audit|IPADDR|WEBHDFS||||access|uri|/gateway/default/webhdfs/v1|success|Response status: 401

this is my topology configuration:

        <topology>


            <gateway>


                <provider>
                    <role>authentication</role>
                    <name>ShiroProvider</name>
                    <enabled>true</enabled>


            <param>
                <name>sessionTimeout</name>
                <value>15</value>
            </param>            


            <param>
                <name>main.ldapRealm</name>
                <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm</value>
            </param>


            <param>
                <name>main.ldapContextFactory</name>
                <value>org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory</value>
            </param>


            <param>
                <name>main.ldapRealm.contextFactory</name>
                <value>$ldapContextFactory</value>
            </param>


            <param>
                <name>main.ldapRealm.contextFactory.url</name>
                <value>ldap://ragaca.com:389</value>
            </param>


            <param>
                <name>main.ldapRealm.authorizationEnabled</name>
                <value>true</value>
            </param>


            <param>
                <name>main.ldapRealm.contextFactory.authenticationMechanism</name>
                <value>simple</value>
            </param>


            <param>
                <name>main.ldapRealm.userDnTemplate</name>
                <value>sAMAccountName={0}</value>
            </param>


            <param>
                <name>main.ldapRealm.userSearchAttributeName</name>
                <value>sAMAccountName</value>
            </param>


            <param>
                <name>main.ldapRealm.userObjectClass</name>
                <value>person</value>
            </param>


            <param>
                <name>main.ldapRealm.contextFactory.systemUsername</name>
                <value>CN=testUser,OU=testUsers,DC=ragaca,DC=com</value>
            </param>


            <param>
                <name>main.ldapRealm.contextFactory.systemPassword</name>
                <value>*********</value>
            </param>


            <param>
                <name>main.ldapRealm.searchBase</name>
                <value>OU=Domain Users & Groups,DC=ragaca,DC=com</value>
            </param>


            <param>
                <name>main.ldapRealm.userSearchBase</name>
                <value>Users,OU=Domain Users & Groups,DC=ragaca,DC=com</value>
            </param>


            <param>
                <name>main.ldapRealm.userSearchScope</name>
                <value>subtree</value>
            </param>


            <param>
                <name>main.ldapRealm.groupSearchBase</name>
                <value>OU=Groups,OU=Domain Users & Groups,DC=ragaca,DC=com</value>
            </param>


            <param>
                <name>main.ldapRealm.groupObjectClass</name>
                <value>group</value>
            </param>


            <param>
                <name>main.ldapRealm.memberAttribute</name>
                <value>member</value>
            </param>




            <param>
                <name>urls./**</name>
                <value>authcBasic</value>
            </param>


                </provider>


                <provider>
                    <role>identity-assertion</role>
                    <name>Default</name>
                    <enabled>true</enabled>
                </provider>


                <provider>
                    <role>authorization</role>
                    <name>AclsAuthz</name>
                    <enabled>true</enabled>
                </provider>


            </gateway>


            <service>
                <role>NAMENODE</role>
                <url>hdfs://namenode1.ragaca.com:8020</url>
            </service>


            <service>
                <role>JOBTRACKER</role>
                <url>rpc://jt.ragaca.com:8050</url>
            </service>


            <service>
                <role>WEBHDFS</role>
                <url>http://namenode1.ragaca.com:50070/</url>
                <url>http://namenode2.ragaca.com:50070/</url>
            </service>

        </topology>

I also have hadoop.proxyuser.knox.hosts=* and hadoop.proxyuser.knox.groups=* in the core-site of the HDFS configuration

could anyone guess what am I missing

Thank you very much and happy new year

[#2]

@Shota Akhalaia, can you try below code block in your topology,

<service>
  <role>WEBHDFS</role>
  <url>http://namenode1.ragaca.com:50070/webhdfs</url>
</service>

Refer this link.

[#3]

@mvaradkar thank you

tryed but same 401 status in the logs

btw after I enter url in the internet browser (h t t p s :// knox . ragaca . com : 8443/gateway/default/webhdfs/v1) there is 401 not only when I enter my real existing AD username and password but when I enter random symbols in the login prompt there are same "response status 401" in the gateway-audit.log every time

[#4]

Can check main.ldapRealm.contextFactory.systemPassword value in your topology, refer link.

[#5]

can you correct the user search base seems to be incorrect.
Refer : Using Apache Knox with ActiveDirector

             <param>
                <name>main.ldapRealm.userSearchBase</name>
                <value>Users,OU=Domain Users & Groups,DC=ragaca,DC=com</value>
            </param>

[#6]

userSearchBase system usernames and passwords are correct, I copied them from working shiro.ini of zeppelin service

[#7]

Is it possible to share the ldapsearch output for a specific user you're trying to access webhdfs.

or use main.ldapRealm.userSearchBase=OU=Domain Users & Groups,DC=ragaca,DC=com and let me know if it works

[#8]

Hi Shota,

Have you fixed your problem ?

I am currently facing same issue.

Thx.