Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

KNOX Pac4j connection via CURL

KNOX Pac4j connection via CURL

<br>

I am trying to replicate KNOX gateway authentication with KNOX SSO and pac4j provider. I am following this link: https://svn.apache.org/repos/asf/knox/site/books/knox-0-9-0/dev-guide.html

I setup KNOX SSO with pac4j provider with test basic auth. I followed the same steps for 'curl'-ing through the URLs, however in the last step while I send the pac4j cookie to the URL, it shows an error saying problem accessing gateway/sso.

The same experiment works perfectly fine from browser access.

Any suggestions as to what I would be missing in the CURL?

Workflow:

curl -iku abc:abc 'https://localhost:8442/gateway/dapClusterSSO/webhdfs/v1/user?op=LISTSTATUS'

HTTP/1.1 302 Found
Date: Fri, 09 Feb 2018 11:38:08 GMT
Location: https://localhost:8442/gateway/knoxsso/api/v1/websso?originalUrl=https://localhost:8442/gateway/dapC...
Content-Length: 0
Server: Jetty(9.2.15.v20160210)
curl -iku abc:abc 'https://localhost:8442/gateway/knoxsso/api/v1/websso?originalUrl=https://localhost:8442/gateway/dapClusterSSO/webhdfs/v1/user?op=LISTSTATUS'

HTTP/1.1 302 Found
Date: Fri, 09 Feb 2018 11:40:06 GMT
Set-Cookie: pac4j.session.pac4jRequestedUrl=AAAACAAAABAAAACgKA1Sf1IwVCKgK0BhCk58OD4OXcA35fLo72qgxzeyoEBWPbqYHJgcYGtt2Fdrs6xTYeU6JMdfnGAG/jT5z+ovN6bJ09bvG4QCo7dpxOFML0ssI8C0AX6JIU/9FWu50IK/Z9FaRWRYmrFEdKRXnzvbhdXzq4uUDvWW2WfylAwgrh3o3jP3b9CqBBN9/ElI//VMrOWO1wTcQaX4a3qz5FxivxqJyu0/UPeo6IWYFILwVxJeKJkEMRvpXw==;Domain=localhost;Secure;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Location: https://localhost:8442/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=IndirectBasicAut...
Content-Length: 0
Server: Jetty(9.2.15.v20160210)
curl -ikH 'Cookie: pac4j.session.pac4jRequestedUrl=AAAACAAAABAAAACgKA1Sf1IwVCKgK0BhCk58OD4OXcA35fLo72qgxzeyoEBWPbqYHJgcYGtt2Fdrs6xTYeU6JMdfnGAG/jT5z+ovN6bJ09bvG4QCo7dpxOFML0ssI8C0AX6JIU/9FWu50IK/Z9FaRWRYmrFEdKRXnzvbhdXzq4uUDvWW2WfylAwgrh3o3jP3b9CqBBN9/ElI//VMrOWO1wTcQaX4a3qz5FxivxqJyu0/UPeo6IWYFILwVxJeKJkEMRvpXw==;Domain=localhost;Secure;HttpOnly' https://localhost:8442/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=IndirectBasicAut...
[1] 100948  
HTTP/1.1 500 Server Error 
Date: Fri, 09 Feb 2018 11:43:19 GMT 
Content-Type: text/html; charset=ISO-8859-1 
Cache-Control: must-revalidate,no-cache,no-store 
Content-Length: 319 
Connection: close 
Server: Jetty(9.2.15.v20160210) 

<html> 
<head> 
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> 
<title>Error 500 Server Error</title> 
</head> 
<body><h2>HTTP ERROR 500</h2> 
<p>Problem accessing /gateway/knoxsso/api/v1/websso. Reason: 
<pre>    Server Error</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/> 
</body>
</html> [1]+  Done                   curl -ikH 'Cookie: pac4j.session.pac4jRequestedUrl=AAAACAAAABAAAACgKA1Sf1IwVCKgK0BhCk58OD4OXcA35fLo72qgxzeyoEBWPbqYHJgcYGtt2Fdrs6xTYeU6JMdfnGAG/jT5z+ovN6bJ09bvG4QCo7dpxOFML0ssI8C0AX6JIU/9FWu50IK/Z9FaRWRYmrFEdKRXnzvbhdXzq4uUDvWW2WfylAwgrh3o3jP3b9CqBBN9/ElI//VMrOWO1wTcQaX4a3qz5FxivxqJyu0/UPeo6IWYFILwVxJeKJkEMRvpXw==;Domain=localhost;Secure;HttpOnly' https://localhost:8442/gateway/knoxsso/api/v1/websso?pac4jCallback=true
7 REPLIES 7
Highlighted

Re: KNOX Pac4j connection via CURL

Contributor

Hello @Nisha

It is difficult to say anything without seeing your topology, but I am assuming you are using "SSOCookieProvider" in your topology. Try using "JWTProvider"

https://knox.apache.org/books/knox-1-0-0/user-guide.html#JWT+Provider

Also check this out

https://knox.apache.org/books/knox-1-0-0/user-guide.html#KnoxToken+Configuration

Best,

Sandeep

Highlighted

Re: KNOX Pac4j connection via CURL

Hello @Sandeep More

Well, I would like to continue using pac4j provider, as I intend to connect to Azure AD later with openID protocol. Infact, I have also done that already. But there is an infinite redirect issue there, and I am not able to debug it. Thats when I wanted to manually curl the basic test Auth first to understand how the cookie can be manually provided to the pac4j callback URL (step 3 in my question above).

My topologies are as follows:

sandbox.xml:

 <topology>
            <gateway>
                <provider>
                    <role>webappsec</role>
                    <name>WebAppSec</name>
                    <enabled>true</enabled>
                    <param>
                        <name>cors.enabled</name>
                        <value>true</value>
                    </param>
                </provider>
                <provider>
                    <role>federation</role>
                    <name>SSOCookieProvider</name>
                    <enabled>true</enabled>
                    <param>
                      <name>sso.authentication.provider.url</name>
                      <value>https://localhost:8442/gateway/knoxsso/api/v1/websso</value>
                    </param>
                </provider>
                <provider>
                    <role>identity-assertion</role>
                    <name>Default</name>
                    <enabled>true</enabled>
                </provider>
            </gateway>
            <service>
                <role>NAMENODE</role>
                <url>hdfs://localhost:8020</url>
            </service>
... // remaining services

knoxsso.xml:

<topology>
          <gateway>
              <provider>
                    <role>federation</role>
                    <name>pac4j</name>
                    <enabled>true</enabled>
                    <param>
                      <name>pac4j.callbackUrl</name>
                      <value>https://localhost:8442/gateway/knoxsso/api/v1/websso</value>
                    </param>
                    <param>
                      <name>clientName</name>
                      <value>testBasicAuth</value>
                    </param>
	      </provider>
              <provider>
                  <role>identity-assertion</role>
                  <name>Default</name>
                  <enabled>true</enabled>
              </provider>
          </gateway>

          <application>
            <name>knoxauth</name>
          </application>

          <service>
              <role>KNOXSSO</role>
              <param>
                  <name>knoxsso.cookie.secure.only</name>
                  <value>true</value>
              </param>
              <param>
                  <name>knoxsso.token.ttl</name>
                  <value>30000</value>
              </param>
              <param>
                 <name>knoxsso.redirect.whitelist.regex</name>
                 <value>^https?:\/\/(localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*{replace1}lt;/value>
              </param>
          </service>
      </topology>

So just to repeat, I would like to see the files listed in webHDFS after the 3rd curl step in the question. Instead I see server error accessing /gateway/knoxsso. And the issue is ONLY for manual curl. The same access via browser works perfectly. (This is because internally all redirection and passing of cookies happens properly in the browser). In manual 'curl', I am guessing I am not passing the cookie properly.

Highlighted

Re: KNOX Pac4j connection via CURL

Contributor

ah, I see, I have had issues with using localhost, to get around this in my etc/host file I add an entry for www.local.com -> 127.0.0.1

and in your topology add it to the whitelist

<param>
<name>knoxsso.redirect.whitelist.regex</name>
<value>^https?:\/\/(www\.local\.com|localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$</value>
</param>

Then try to access it using https://www.local.com:8443/.

This should help, let me know if this does not help.

Best,

Sandeep

Highlighted

Re: KNOX Pac4j connection via CURL

Hello @Sandeep More

Well, no thats not the issue. I had figured out the whitelist importance earlier itself. This is something to do with the CURL request itself. Like I said, from the browser it works fine. So its definitely nothing to do with localhost or IP :(

Highlighted

Re: KNOX Pac4j connection via CURL

@Sandeep More following your advice, I tried JWT provider example and the curl requests passes through successfully. In JWT, it clearly says that it accepts "Authorization : Bearer" token, so passing that to the REST call was straightforward and easy.

However, the SSO issue still remains. The 3rd curl refuses to pass through inspite of providing the "Cookie".

Highlighted

Re: KNOX Pac4j connection via CURL

Contributor

@Nisha that is by design, try passing cookies through Curl with the CookieProvider, I think this should work !

Highlighted

Re: KNOX Pac4j connection via CURL

@Sandeep More, Nope this also didnt work. I tried: "Cookie" "CookieProvider", "Set-Cookie", "pac4j" etc. None of this seems to understand what is being passed to the callback URL.

curl -ikH 'CookieProvider: pac4j.session.pac4jRequestedUrl=AAAACAAAABAAAACgKA1Sf1IwVCKgK0BhCk58OD4OXcA35fLo72qgxzeyoEBWPbqYHJgcYGtt2Fdrs6xTYeU6JMdfnGAG/jT5z+ovN6bJ09bvG4QCo7dpxOFML0ssI8C0AX6JIU/9FWu50IK/Z9FaRWRYmrFEdKRXnzvbhdXzq4uUDvWW2WfylAwgrh3o3jP3b9CqBBN9/ElI//VMrOWO1wTcQaX4a3qz5FxivxqJyu0/UPeo6IWYFILwVxJeKJkEMRvpXw==;Domain=localhost;Secure;HttpOnly' https://localhost:8442/gateway/knoxsso/api/v1/websso?pac4jCallback=true&client_name=IndirectBasicAut...

Did you get it working with curl?

As per the developer docs, https://svn.apache.org/repos/asf/knox/site/books/knox-0-9-0/dev-guide.html#KnoxSSO+Integration, the author does not confirm the results of the last curl passed with hadoop-jwt Cookie.

Regards,

Nisha

Don't have an account?
Coming from Hortonworks? Activate your account here