@mqureshi I have a similar problem but in my case I dont want to create separate tickets for application users. My requirement is that all services in Hadoop should be accessed via KNOX as proxy user. Knox would have taken care of authentication seperately. So in my case all authenticated application users eg user1, user2 etc.. should be able to run jobs with knox proxy user. This link talks exactly of the same concept: https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/Superusers.html#Use_Case Here the idea is not to have seperate kerberos credentials for each individual application users. Any thoughts from your side, on what would be required for this? ... View more

@Sandeep More, Nope this also didnt work. I tried: "Cookie" "CookieProvider", "Set-Cookie", "pac4j" etc. None of this seems to understand what is being passed to the callback URL. curl -ikH 'CookieProvider: pac4j.session.pac4jRequestedUrl=AAAACAAAABAAAACgKA1Sf1IwVCKgK0BhCk58OD4OXcA35fLo72qgxzeyoEBWPbqYHJgcYGtt2Fdrs6xTYeU6JMdfnGAG/jT5z+ovN6bJ09bvG4QCo7dpxOFML0ssI8C0AX6JIU/9FWu50IK/Z9FaRWRYmrFEdKRXnzvbhdXzq4uUDvWW2WfylAwgrh3o3jP3b9CqBBN9/ElI//VMrOWO1wTcQaX4a3qz5FxivxqJyu0/UPeo6IWYFILwVxJeKJkEMRvpXw==;Domain=localhost;Secure;HttpOnly' https://localhost:8442/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=IndirectBasicAuthClient Did you get it working with curl? As per the developer docs, https://svn.apache.org/repos/asf/knox/site/books/knox-0-9-0/dev-guide.html#KnoxSSO+Integration, the author does not confirm the results of the last curl passed with hadoop-jwt Cookie. Regards, Nisha ... View more

@Sandeep More following your advice, I tried JWT provider example and the curl requests passes through successfully. In JWT, it clearly says that it accepts "Authorization : Bearer" token, so passing that to the REST call was straightforward and easy. However, the SSO issue still remains. The 3rd curl refuses to pass through inspite of providing the "Cookie". ... View more

Hello @Sandeep More Well, no thats not the issue. I had figured out the whitelist importance earlier itself. This is something to do with the CURL request itself. Like I said, from the browser it works fine. So its definitely nothing to do with localhost or IP 😞 ... View more

Hello @Sandeep More Well, I would like to continue using pac4j provider, as I intend to connect to Azure AD later with openID protocol. Infact, I have also done that already. But there is an infinite redirect issue there, and I am not able to debug it. Thats when I wanted to manually curl the basic test Auth first to understand how the cookie can be manually provided to the pac4j callback URL (step 3 in my question above). My topologies are as follows: sandbox.xml: <topology> <gateway> <provider> <role>webappsec</role> <name>WebAppSec</name> <enabled>true</enabled> <param> <name>cors.enabled</name> <value>true</value> </param> </provider> <provider> <role>federation</role> <name>SSOCookieProvider</name> <enabled>true</enabled> <param> <name>sso.authentication.provider.url</name> <value>https://localhost:8442/gateway/knoxsso/api/v1/websso</value> </param> </provider> <provider> <role>identity-assertion</role> <name>Default</name> <enabled>true</enabled> </provider> </gateway> <service> <role>NAMENODE</role> <url>hdfs://localhost:8020</url> </service> ... // remaining services knoxsso.xml: <topology> <gateway> <provider> <role>federation</role> <name>pac4j</name> <enabled>true</enabled> <param> <name>pac4j.callbackUrl</name> <value>https://localhost:8442/gateway/knoxsso/api/v1/websso</value> </param> <param> <name>clientName</name> <value>testBasicAuth</value> </param> </provider> <provider> <role>identity-assertion</role> <name>Default</name> <enabled>true</enabled> </provider> </gateway> <application> <name>knoxauth</name> </application> <service> <role>KNOXSSO</role> <param> <name>knoxsso.cookie.secure.only</name> <value>true</value> </param> <param> <name>knoxsso.token.ttl</name> <value>30000</value> </param> <param> <name>knoxsso.redirect.whitelist.regex</name> <value>^https?:\/\/(localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*{replace1}lt;/value> </param> </service> </topology> So just to repeat, I would like to see the files listed in webHDFS after the 3rd curl step in the question. Instead I see server error accessing /gateway/knoxsso. And the issue is ONLY for manual curl. The same access via browser works perfectly. (This is because internally all redirection and passing of cookies happens properly in the browser). In manual 'curl', I am guessing I am not passing the cookie properly. ... View more

@Aditya Sirna Is there a way to add the SQL commands to the blueprint, (or reference to a script file) that should be run on MySQL once MySQL has been installed. Without the blueprint, there is no MySQL installed, so the commands should also be part of the blueprint if thats possible. ... View more

@gnovak @tuxnet Would resource sharing still work if ACLs are configured for separate tenant queues? If ACLs are different for Q1 and Q2, will it still support elasticity and preemption? Could you also please share the workload/application details that you used for these experiments? I am trying to run some experiments to do a similar test for elasticity and preemption of capacity schedulers. I am using a simple Spark word count application on a large file for the same, but I am not able to get a feel of resource sharing among queues using this application. Thanks in advance. ... View more