Created 01-03-2018 05:22 AM
Issue: while accessing knox gateway through curl throws error as "HTTP ERROR 404"
Below details of cluster.
Ambari: 2.6
HDP: 2.6.3
Authentication: Windows AD with kerberos
Authorization: Ranger
Knox integrated with Ranger and Knox Repo on Ranger UI Test connection is success.
Knox default topology configured with windows AD credentials. Below snap for the same.
[root@vijayhdp-1 topologies]# pwd
/etc/knox/conf/topologies
[root@vijayhdp-1 topologies]# ls -rlt default.xml
-rw-r--r-- 1 knox knox 4442 Dec 22 13:58 default.xml
[root@vijayhdp-1 topologies]# cat default.xml
<topology> <gateway> <provider> <role>authentication</role> <name>ShiroProvider</name> <enabled>true</enabled> <param name="sessionTimeout value=30/> <param name="main.ldapRealm" value="org.apache.hadoop.gateway.shirorealm.KnoxLdapRealm"/> <param name="main.ldapContextFactory" value="org.apache.hadoop.gateway.shirorealm.KnoxLdapContextFactory"/> <param name="main.ldapRealm.contextFactory" value="$ldapContextFactory"/> <param name="main.ldapRealm.contextFactory.url" value="ldaps://HADOOP-AD.HADOOP.LOCAL:636"/> <param name="main.ldapRealm.contextFactory.systemUsername" value="CN=Admin account,OU=HADOOP,DC=HADOOP,DC=LOCAL"/> <param name="main.ldapRealm.contextFactory.systemPassword" value="Passw0rd"/> <param name="main.ldapRealm.userSearchBase" value="OU=HADOOP,DC=HADOOP,DC=LOCAL"/> <param name="main.ldapRealm.userSearchAttributeName" value="sAMAccountName"/> <param name="main.ldapRealm.userObjectClass" value="person"/> <param name="main.ldapRealm.authorizationEnabled" value="true"/> <param name="main.ldapRealm.groupSearchBase" value="OU=HADOOP,DC=HADOOP,DC=LOCAL"/> <param name="main.ldapRealm.groupObjectClass" value="group"/> <param name="main.ldapRealm.groupIdAttribute" value="sAMAccountName"/> <param name="main.ldapRealm.memberAttribute" value="member"/> <param name="main.cacheManager" value="org.apache.shiro.cache.ehcache.EhCacheManager"/> <param name="main.securityManager.cacheManager" value="$cacheManager"/> <param name="main.ldapRealm.authenticationCachingEnabled" value="true"/>
While accessing the gateway it throws below error.
[root@vijayhdp-1 ~]# curl -iku ajay:Passw0rd -X GET 'https://vijayhdp-1.novalocal:8443/gateway/default/webhdfs/v1?op=LISTSTATUS' HTTP/1.1 404 Not Found Date: Wed, 03 Jan 2018 05:22:15 GMT Cache-Control: must-revalidate,no-cache,no-store Content-Type: text/html; charset=ISO-8859-1 Content-Length: 304 Server: Jetty(9.2.15.v20160210) <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /gateway/default/webhdfs/v1. Reason: <pre> Not Found</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html>
Kindly help me to fix the issue.
- Vijay Mishra
Created 01-03-2018 05:45 AM
Does your deafult.xml has just 2 lines as given in cat command output?
Created 01-03-2018 07:14 AM
Did some troubleshooting and found issue default topology, knox unable to activate default topology. I have corrected the issue and 404 error gone. But started getting new error i.e. 403
[ajay@vijayhdp-1 ~]$ curl -iku ajay:Passw0rd -X GET 'https://vijayhdp-1.novalocal:8443/gateway/default/webhdfs/v1/user/ajay?op=LISTSTATUS' HTTP/1.1 403 Forbidden Date: Wed, 03 Jan 2018 07:13:48 GMT Set-Cookie: JSESSIONID=1hgp22be0fp121chqv5fmgei6q;Path=/gateway/default;Secure;HttpOnly Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; Expires=Tue, 02-Jan-2018 07:13:48 GMT Content-Type: text/html; charset=ISO-8859-1 Cache-Control: must-revalidate,no-cache,no-store Content-Length: 320 Server: Jetty(9.2.15.v20160210) <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> <title>Error 403 Forbidden</title> </head> <body><h2>HTTP ERROR 403</h2> <p>Problem accessing /gateway/default/webhdfs/v1/user/ajay. Reason: <pre> Forbidden</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/> </body> </html> [ajay@vijayhdp-1 ~]$
Knox gateway log:
2018-01-03 12:43:48,296 INFO hadoop.gateway (KnoxLdapRealm.java:getUserDn(724)) - Computed userDn: CN=ajay,OU=HADOOP,DC=HADOOP,DC=LOCAL using ldapSearch for principal: ajay 2018-01-03 12:43:48,506 INFO hadoop.gateway (KnoxLdapRealm.java:getUserDn(724)) - Computed userDn: CN=ajay,OU=HADOOP,DC=HADOOP,DC=LOCAL using ldapSearch for principal: ajay 2018-01-03 12:43:48,528 INFO hadoop.gateway (KnoxLdapRealm.java:rolesFor(327)) - Computed roles/groups: [app] for principal: ajay
Kindly suggest.
- Vijay Mishra
Created 01-03-2018 07:20 AM
Can you remove authorization provider from default topology and see if it's due to Ranger policies preventing access?
Created 01-04-2018 03:59 AM
Krishna Pandey
Yes, Issue was with authorization and its fixed now.
- Vijay Mishra