We use the following topologies right now in HDP 3, shown below. I understand there is authentication and authorization. Prior to using PAM, we used LDAP configurations. Then we ran into...
- KnoxLdapContextFactory.java (HW had us try identity assertion, null pointer issue hit)
- KnoxLdapRealm.java (ldap - too many entries, fails, we only get the 'cdisadmin' group as it's in the first page or results, and the code for this does NOT properly page)
- KnoxPamRealm.java (works, but requires reworking PAM/sssd,).
PAM works it seems, but even if I put my own username in the top level Knox policy to deny access, nothing is denied when I access Knox SSO webui's. I don't understand what part of the authorization process is not communicating. In our view, when you pass the PAM authentication stage, you just get access, when what we REALLY want to happen, is Ranger then says yes or no to you getting authorized.
PAM module:
#%PAM-1.0
auth [user_unknown=ignore success=ok ignore=ignore default=bad] pam_securetty.so
#auth substack system-auth
###############################################
# Imported from: /etc/pam.d/password-auth
###############################################
auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth sufficient pam_ldap.so minimum_uid=1000 use_first_pass
auth requisite pam_succeed_if.so uid >= 1000 quiet
#auth sufficient pam_sss.so use_first_pass
auth required pam_deny.so
account required pam_unix.so broken_shadow
account sufficient pam_ldap.so minimum_uid=1000
account sufficient pam_localuser.so
account sufficient pam_succeed_if.so uid < 1000 quiet
account [default=bad success=ok user_unknown=ignore] pam_sss.so
account required pam_permit.so
password requisite pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type=
password sufficient pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password sufficient pam_ldap.so minimum_uid=1000 try_first_pass
#password sufficient pam_sss.so use_authtok
password required pam_deny.so
session optional pam_keyinit.so revoke
session required pam_limits.so
session optional pam_mkhomedir.so umask=0077
-session optional pam_systemd.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
session optional pam_ldap.so minimum_uid=1000
#session optional pam_sss.so
knoxsso topology
<topology>
<gateway>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
<provider>
<role>webappsec</role>
<name>WebAppSec</name>
<enabled>true</enabled>
<param>
<name>xframe.options.enabled</name>
<value>true</value>
</param>
</provider>
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<param>
<name>sessionTimeout</name>
<value>30</value>
</param>
<param>
<name>redirectToUrl</name>
<value>/gateway/knoxsso/knoxauth/login.html</value>
</param>
<param>
<name>restrictedCookies</name>
<value>rememberme,WWW-Authenticate</value>
</param>
<param>
<name>main.pamRealm</name>
<value>org.apache.knox.gateway.shirorealm.KnoxPamRealm</value>
</param>
<param>
<name>main.pamRealm.service</name>
<value>knoxsso</value>
</param>
<param>
<name>urls./**</name>
<value>authcBasic</value>
</param>
</provider>
<provider>
<role>authorization</role>
<name>AclsAuthz</name>
<enabled>true</enabled>
</provider>
</gateway>
<application>
<name>knoxauth</name>
</application>
<service>
<role>KNOXSSO</role>
<param>
<name>knoxsso.cookie.secure.only</name>
<value>true</value>
</param>
<param>
<name>knoxsso.token.ttl</name>
<value>900000</value>
</param>
<param>
<name>knoxsso.redirect.whitelist.regex</name>
<value>^https?:\/\/(.*\.DOMAIN\.COM|localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$</value>
</param>
</service>
</topology>
default topology:
<topology>
<gateway>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<param>
<name>sessionTimeout</name>
<value>30</value>
</param>
<param>
<name>main.pamRealm</name>
<value>org.apache.hadoop.gateway.shirorealm.KnoxPamRealm</value>
</param>
<param>
<name>main.pamRealm.service</name>
<value>knoxsso</value> </param>
<param>
<name>urls./**</name>
<value>authcBasic</value>
</param>
</provider>
<provider>
<role>authorization</role>
<name>XASecurePDPKnox</name>
<enabled>true</enabled>
</provider>
</gateway>
<service>
<role>AVATICA</role>
<url>http://HOST.DOMAIN.COM:8765</url>
</service>
<service>
<role>DRUID-COORDINATOR-UI</role>
{{druid_coordinator_urls}}
</service>
<service>
<role>DRUID-COORDINATOR</role>
{{druid_coordinator_urls}}
</service>
<service>
<role>DRUID-OVERLORD-UI</role>
{{druid_overlord_urls}}
</service>
<service>
<role>DRUID-OVERLORD</role>
{{druid_overlord_urls}}
</service>
<service>
<role>DRUID-ROUTER</role>
{{druid_router_urls}}
</service>
<service>
<role>DRUID-BROKER</role>
{{druid_broker_urls}}
</service>
<service>
<role>HBASEUI</role>
<url>http://HOST.DOMAIN.COM:16010</url>
</service>
<service>
<role>HDFSUI</role>
<version>2.7.0</version>
<url>http://HOST.DOMAIN.COM:50070/</url>
</service>
<service>
<role>HIVE</role>
<url>http://{{hive_server_host}}:{{hive_http_port}}/{{hive_http_path}}</url>
</service>
<service>
<role>JOBTRACKER</role>
<url>rpc://{{rm_host}}:{{jt_rpc_port}}</url>
</service>
<service>
<role>JOBHISTORYUI</role>
<url>http://HOST.DOMAIN.COM:19888</url>
</service>
<service>
<role>NAMENODE</role>
<url>{{namenode_address}}</url>
</service>
<service>
<role>OOZIE</role>
<url>http://{{oozie_server_host}}:{{oozie_server_port}}/oozie</url>
</service>
<service>
<role>OOZIEUI</role>
<url>http://{{oozie_server_host}}:{{oozie_server_port}}/oozie/</url>
</service>
<service>
<role>RANGERUI</role>
<url>http://HOST.DOMAIN.COM:6080</url>
</service>
<service>
<role>RESOURCEMANAGER</role>
<url>http://{{rm_host}}:{{rm_port}}/ws</url>
</service>
<service>
<role>SPARKHISTORYUI</role>
<url>http://HOST.DOMAIN.COM:18081</url>
</service>
<service>
<role>WEBHDFS</role>
{{webhdfs_service_urls}}
</service>
<service>
<role>WEBHCAT</role>
<url>http://{{webhcat_server_host}}:{{templeton_port}}/templeton</url>
</service>
<service>
<role>WEBHBASE</role>
<url>http://{{hbase_master_host}}:60080</url>
</service>
<service>
<role>YARNUI</role>
<url>http://HOST.DOMAIN.COM:8088</url>
</service>
<service>
<role>YARNUIV2</role>
<url>http://HOST.DOMAIN.COM:8088</url>
</service>
<service>
<role>ZEPPELINUI</role>
{{zeppelin_ui_urls}}
</service>
<service>
<role>ZEPPELINWS</role>
{{zeppelin_ws_urls}}
</service>
</topology>
This is all I noticed in gateway.log when testing these scenarios:
[mtdeguzis@HOST: knox]$ tailf gateway.log | grep -iE 'mtdeguzis|yarn'
2019-03-22 15:49:07,094 INFO knox.gateway (KnoxPamRealm.java:doGetAuthorizationInfo(126)) - Computed roles/groups: [udaoptst3, udaops, HOST-login, mtdeguzis, cdisadmin] for principal: mtdeguzis
2019-03-22 15:49:07,405 INFO service.knoxsso (WebSSOResource.java:getAuthenticationToken(240)) - About to redirect to original URL: https://HOST.domain.com:8443/gateway/knoxsso-webuis/yarnuiv2
2019-03-22 15:49:07,972 ERROR knox.gateway (UrlRewriteProcessor.java:rewrite(166)) - Failed to rewrite URL: HTTP_ONLY, direction: OUT via rule: YARNUIV2/yarnuiv2/outbound/timeline, status: FAILURE
2019-03-22 15:49:07,973 ERROR knox.gateway (JsonFilterReader.java:filterStreamValue(547)) - Failed to filter value HTTP_ONLY, rule YARNUIV2/yarnuiv2/outbound/timeline: java.lang.NullPointerException
2019-03-22 15:49:08,013 ERROR knox.gateway (UrlRewriteProcessor.java:rewrite(166)) - Failed to rewrite URL: kerberos, direction: OUT via rule: YARNUIV2/yarnuiv2/outbound/timeline, status: FAILURE
2019-03-22 15:49:08,013 ERROR knox.gateway (JsonFilterReader.java:filterStreamValue(547)) - Failed to filter value kerberos, rule YARNUIV2/yarnuiv2/outbound/timeline: java.lang.NullPointerException
