HttpFS Pam Configuration

[#1]

I am trying to use HttpFS with PAM authentication. I was able to install it using this post: https://community.hortonworks.com/content/kbentry/804/httpfs-configure-and-run-with-hdp-224x.html

Then I tried to configure PAM authentication using JPam. I copied the jpam-1.1.jar under /usr/hdp/current/hadoop-httpfs/webapps/webhdfs/WEB-INF/lib

I also modified /usr/hdp/current/hadoop-httpfs/webapps/webhdfs/META-INF/context.xml to be

<Context><Realm className="org.apache.catalina.realm.JAASRealm" appName="jpamLogin"/></Context>

everytime I try to access httpfs and provide the username:password it gives me this error

org.apache.catalina.realm.JAASRealm authenticateSEVERE: Unexpected errorjavax.security.auth.login.LoginException: No LoginModules configured for jpamLoginat javax.security.auth.login.LoginContext.init(LoginContext.java:264)at javax.security.auth.login.LoginContext.<init>(LoginContext.java:417)at org.apache.catalina.realm.JAASRealm.authenticate(JAASRealm.java:393)at org.apache.catalina.realm.JAASRealm.authenticate(JAASRealm.java:334)at org.apache.catalina.authenticator.BasicAuthenticator.authenticate(BasicAuthenticator.java:181)at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:528)at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:610)at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:503)at java.lang.Thread.run(Thread.java:748)

UPDATE 2:

I made some progress. I created the file /etc/hadoop-httpfs/tomcat-deployment/conf/jaas.config with the following content

jpamLogin {net.sf.jpam.jaas.JpamLoginModule required serviceName="password-auth";};

then I updated httpfs-env.sh with

export CATALINA_OPTS='-Djava.security.auth.login.config=/etc/hadoop-httpfs/tomcat-deployment/conf/jaas.config -Djava.library.path=/usr/hdp/current/hadoop-httpfs/webapps/webhdfs/WEB-INF/lib/'

Restart httpfs and I get this exception now.

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000; background-color: #ffffff}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000; background-color: #ffffff; min-height: 13.0px}
span.s1 {font-variant-ligatures: no-common-ligatures}
span.Apple-tab-span {white-space:pre}



javax.security.auth.login.LoginException: java.lang.UnsatisfiedLinkError: no jpam in java.library.path

	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)

	at java.lang.Runtime.loadLibrary0(Runtime.java:870)

	at java.lang.System.loadLibrary(System.java:1122)

	at net.sf.jpam.Pam.<clinit>(Pam.java:51)

	at net.sf.jpam.jaas.JpamLoginModule.createPam(JpamLoginModule.java:171)

	at net.sf.jpam.jaas.JpamLoginModule.login(JpamLoginModule.java:126)

	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

	at java.lang.reflect.Method.invoke(Method.java:498)

	at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)

	at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)

	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)

	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)

	at java.security.AccessController.doPrivileged(Native Method)

	at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)

	at javax.security.auth.login.LoginContext.login(LoginContext.java:587)

	at org.apache.catalina.realm.JAASRealm.authenticate(JAASRealm.java:409)

	at org.apache.catalina.realm.JAASRealm.authenticate(JAASRealm.java:334)

	at org.apache.catalina.authenticator.BasicAuthenticator.authenticate(BasicAuthenticator.java:181)

	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:528)

	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)

	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)

	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)

	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)

	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:610)

	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:503)

	at java.lang.Thread.run(Thread.java:748)




	at javax.security.auth.login.LoginContext.invoke(LoginContext.java:856)

	at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)

	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)

	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)

	at java.security.AccessController.doPrivileged(Native Method)

	at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)

	at javax.security.auth.login.LoginContext.login(LoginContext.java:587)

	at org.apache.catalina.realm.JAASRealm.authenticate(JAASRealm.java:409)

	at org.apache.catalina.realm.JAASRealm.authenticate(JAASRealm.java:334)

	at org.apache.catalina.authenticator.BasicAuthenticator.authenticate(BasicAuthenticator.java:181)

	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:528)

	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)

	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)

	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)

	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)

	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:610)

	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:503)

	at java.lang.Thread.run(Thread.java:748)

I appreciate your help

[#2]

Are you able to use WebHdfs with Knox and use PAM authentication in Knox? If so, please see https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66854729

[#3]

My cluster is not kerberized and I do not want to go with webhdfs due to the impersonation issue. Httpfs is a better option and would prevent a user from impersonating hdfs.

[#4]

So I was able to get it to work by copying the libjpam.so to the WEB-INF/lib folder and restart httpfs. The problem now is that it requires from me to provide the parameter user.name=<username> as the user performing the action, not the one logging in. Is there a way to eliminate this variable or block it so it is always the logged in user who is performing the actions on hdfs?

[#5]

Using Knox in concert with WebHDFS eliminates the impersonation issue you mention. The user authenticates to Knox, and Knox securely impersonates the user using the doAs API when connecting to HDP services.

Authenticating at the Knox-level also solves for the user.name issue you mention.

[#6]

Can you limit access to WebHdfs/Httpfs to be only through Knox? If that is possible, how is that accomplished?

[#7]

By limiting which ports users can access from outside of the cluster. You can choose to only expose the Knox HTTPS port, 8443 by default.

Please consider accepting the answer if this is helpful.

[#8]

Do you have an example please?

[#9]

https://hortonworks.com/hadoop-tutorial/securing-hadoop-infrastructure-apache-knox/