Microservice applications prefer OpenID SSO by existing IDP, for example Keycloak. When these application access Hadoop Services with JWT, Knox should be able to verify and proxy to internal services. This article describe how to configure Knox JWT Provider and customize for your IDP requirement.
There are two options.
KnoxSSO direct integrate with IDP
KnoxSSO, Knox Single Sign On service, integrates with IDP, for example Keycloak. After authenticated by IDP, Knox signs a JWT token for all microservices and Hadoop RESTful services. This solution is complex to configure and Knox could become the performance bottleneck.
Knox JWTProvider accept jwt signed by other IDP
Microservices still use current IDP for single sign on, and configure an extra Knox topology to accept other IDP signed JWT token. The second solution has less impact on existing architecture and Knox is only used when access Hadoop services. Here we will focus on this solution, and discuss how to configure Knox and what is the limitation.
Add a JWTProvider Knox topology
JWTProvider is explained
https://knox.apache.org/books/knox-0-12-0/user-guide.html#JWT+Provider
Current Ambari can't edit extra Knox topology, have to add it in command line.
# ssh Knox node $ sudo su - knox # Add the new topology named jwt $ vim jwt.xml
<topology>
<gateway>
<provider>
<role>identity-assertion</role>
<name>Default</name>
<enabled>true</enabled>
</provider>
<provider>
<role>authorization</role>
<name>XASecurePDPKnox</name>
<enabled>true</enabled>
</provider>
<provider>
<role>federation</role>
<name>JWTProvider</name>
<enabled>true</enabled>
<param>
<!-- knox.token.audiences is optional -->
<name>knox.token.audiences</name>
<value>tokenbased</value>
</param>
</provider>
</gateway>
<!-- Add Hadoop Services allowed jwt access, here use Yarn UI as an example -->
<service>
<role>YARNUI</role>
<url>http://{MASTER_NODE_1}:8088</url>
<url>http://{MASTER_NODE_2}:8088</url>
</service>
<service>
<role>RESOURCEMANAGER</role>
<url>http://{MASTER_NODE_1}:8088/ws</url>
<url>http://{MASTER_NODE_2}:8088/ws</url>
</service>
</topology>Add Knox Token Service in knoxsso toplogy for test
<service>
<role>KNOXTOKEN</role>
<param>
<name>knox.token.ttl</name>
<value>600000</value>
</param>
<!-- knox.token.audiences is optional, must meet what's configured in JWTProvider -->
<param>
<name>knox.token.audiences</name>
<value>tokenbased</value>
</param>
</service>Test Knox jwt toplogy
Enable Knox demo ldap as knoxsso source accounts.
Get a knoxsso jwt token.
$ curl -ivku guest:guest-password https://{KNOX_NODE}:8443/gateway/knoxsso/knoxtoken/api/v1/token {"access_token":"eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJndWVzdCIsImF1ZCI6InRva2VuYmFzZWQiLCJpc3MiOiJLTk9YU1NPIiwiZXhwIjoxNTI4MjgxODQ3fQ.i2Y3MMGbTW9M-wHStL7TuwjmL_rYmTGRjN_7QK0KB8EfLxKJzL2zRFEU8USxyFAchDJ-3vDdLaU8UPsTCVufo9UT5p8ywSlBgulFsOzIYuq-YVIqATpJZVZIJWCnoGHjXuTZHXeRreyjAs6cFsiiqsDwL8rCxnAmtBQeoX9fsAI","token_type":"Bearer ","expires_in":1528281847804}
Use this token to access Yarn UI
$ curl -ivk -H "Authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJndWVzdCIsImF1ZCI6InRva2VuYmFzZWQiLCJpc3MiOiJLTk9YU1NPIiwiZXhwIjoxNTI4MjgxODQ3fQ.i2Y3MMGbTW9M-wHStL7TuwjmL_rYmTGRjN_7QK0KB8EfLxKJzL2zRFEU8USxyFAchDJ-3vDdLaU8UPsTCVufo9UT5p8ywSlBgulFsOzIYuq-YVIqATpJZVZIJWCnoGHjXuTZHXeRreyjAs6cFsiiqsDwL8rCxnAmtBQeoX9fsAI" https://hdp-e.field.hortonworks.com:8443/gateway/jwt/yarn/
admin should be able to view the UI html contents, and guest would be blocked. Also can find the entries in Ranger Audit.
Decode knoxsso jwt token
Decode the jwt https://jwt.io/
HEADER
{
"alg": "RS256"
}
PAYLOAD
{
"sub": "guest",
"aud": "tokenbased",
"iss": "KNOXSSO",
"exp": 1528281847
}Microservice Application JWT Verification
Synchronise certificates between Knox and IDP
IDP use private certificate to sign JWT, and Knox use public certificate to verify the JWT. The certificate pair must match for the verification. In current Knox version, public key is not supported, have to configure public certificate as following example.
Knox only supports keypair JWT signature algorithms, RS256, RS384, RS512, PS256, PS384, PS512. RS256 is default and not configurable in HDP2.6.
Get public certificate from IDP.
$ cat knox-pub.pem -----BEGIN CERTIFICATE----- MIIENTCCAx2gAwIBAgIJAP4/owzmw1t4MA0GCSqGSIb3DQEBCwUAMIGlMQswCQYD VQQGEwJVSzEABC0GA1UECAwGTG9uZG9uMQ8wDQYDVQQHDAZMb25kb24xFDASBgNV BAoMC0hvbWUgT2ZmaWNlMUQwQgYDVQQDDDtwcmltYXJ5LXNlY3VyaXR5MC5ub25w cm9kZi51ay5zZHAuZGlnaXRhbC5ob21lb2ZmaWNlLmdvdi51azEYMBYGA1UdEQwP RE5TLjE9bG9jYWxob3N0MB4XDTE4MDUzMTE2MTA0MFoXDTI4MDUyODE2MTA0MFow gaUxCzAJBgNVBAYTAlVLMQ8wDQYDVQQIDAZMb25kb24xDzANBgNVBAcMBkxvbmRv bjEUMBIGA1UECgwLSG9tZSBPZmZpY2UxRDBCBgNVBAMMO3ByaW1hcnktc2VjdXJp dHkwLm5vbnByb2RmLnVrLnNkcC5kaWdpdGFsLmhvbWVvZmZpY2UuZ292LnVrMRgw FgYDVR0RDA9ETlMuMT1sb2NhbGhvc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw ggEKAoIBAQDNb8qkTY0afOAUauLWbLZfF/1kys6Il/aRwKLaf0m+nTiaYowCGBKB XLbzkXXDNSCOC2b6qPRIul2yF1fd8hAmnfUiVJY2hKbbfhdvbPp1oF9cC6QL+28x ppj+utmyf2YNYpXGCMKHh7wZHNTTt28jqB/+Co8RC2xQgQ6FX6rCmUSB62/VuMHp JBCb4Sf3Y9qgiZKyqtK9B9UiNIo4bJqeyF3Ql3qCFKmXIoni4k/3wcKvw9wNiWPy MQweK0Q7G542K02Q3h7+caRNnZibFSTC/Qvs5jsRrxmkw4vH3npEJGASVr+y9JGq dCjsi+ocrEdN9SMnGlQpWWtuB8LRlsitAgMBAAGjZjBkMBQGA1UdEQQNMAuCCWxv Y2FsaG9zdDAdBgNVHQ4EFgQUPlLDl7837zNPS2Dvn2u7mtqDTZgwHwYDVR0jBBgw FoAUPlLDl7837zNPS2Dvn2u7mtqDTZgwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0B AQsFAAOCAQEAZa7ZJfc8MzwmYCCmVt9xcGQqFeAxC4saqKEFuS6PJAJqlZIK+D/A y+3AT2qJ84Rs3nWFdnIsEmGzWwLbfve/xyFFAizM8d1tYF5DxXWp+7f2c1Ssah+S t+ua80N9Q2EwdZekQrtnfba58gW5RdTORNGAVjXJjQvHSytwkn1YlRsVQjhvv4Cy R6LBb5Xdd0R9DIqu2mpp0bGvX6hlx0yPJrsiYxd1DsHl+aFdTnQ3OkZVvxe2MfWi yhTIWQoLfHrMwc2l1qjn2c3x4AIRsqLiLkMTfgrgUWC+T2IL1oO5jFBjbeV3ljOY fOfkGmG6TCsdF38qkB/fl869jUGXIBAHjQ== -----END CERTIFICATE-----
In Knox JWT topology, configure the verification public certificate.
<provider>
<role>federation</role>
<name>JWTProvider</name>
<enabled>true</enabled>
<param>
<name>knox.token.verification.pem</name>
<value>
MIIENTCCAx2gAwIBAgIJAP4/owzmw1t4MA0GCSqGSIb3DQEBCwUAMIGlMQswCQYD
VQQGEwJVSzEABC0GA1UECAwGTG9uZG9uMQ8wDQYDVQQHDAZMb25kb24xFDASBgNV
BAoMC0hvbWUgT2ZmaWNlMUQwQgYDVQQDDDtwcmltYXJ5LXNlY3VyaXR5MC5ub25w
cm9kZi51ay5zZHAuZGlnaXRhbC5ob21lb2ZmaWNlLmdvdi51azEYMBYGA1UdEQwP
RE5TLjE9bG9jYWxob3N0MB4XDTE4MDUzMTE2MTA0MFoXDTI4MDUyODE2MTA0MFow
gaUxCzAJBgNVBAYTAlVLMQ8wDQYDVQQIDAZMb25kb24xDzANBgNVBAcMBkxvbmRv
bjEUMBIGA1UECgwLSG9tZSBPZmZpY2UxRDBCBgNVBAMMO3ByaW1hcnktc2VjdXJp
dHkwLm5vbnByb2RmLnVrLnNkcC5kaWdpdGFsLmhvbWVvZmZpY2UuZ292LnVrMRgw
FgYDVR0RDA9ETlMuMT1sb2NhbGhvc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQDNb8qkTY0afOAUauLWbLZfF/1kys6Il/aRwKLaf0m+nTiaYowCGBKB
XLbzkXXDNSCOC2b6qPRIul2yF1fd8hAmnfUiVJY2hKbbfhdvbPp1oF9cC6QL+28x
ppj+utmyf2YNYpXGCMKHh7wZHNTTt28jqB/+Co8RC2xQgQ6FX6rCmUSB62/VuMHp
JBCb4Sf3Y9qgiZKyqtK9B9UiNIo4bJqeyF3Ql3qCFKmXIoni4k/3wcKvw9wNiWPy
MQweK0Q7G542K02Q3h7+caRNnZibFSTC/Qvs5jsRrxmkw4vH3npEJGASVr+y9JGq
dCjsi+ocrEdN9SMnGlQpWWtuB8LRlsitAgMBAAGjZjBkMBQGA1UdEQQNMAuCCWxv
Y2FsaG9zdDAdBgNVHQ4EFgQUPlLDl7837zNPS2Dvn2u7mtqDTZgwHwYDVR0jBBgw
FoAUPlLDl7837zNPS2Dvn2u7mtqDTZgwDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0B
AQsFAAOCAQEAZa7ZJfc8MzwmYCCmVt9xcGQqFeAxC4saqKEFuS6PJAJqlZIK+D/A
y+3AT2qJ84Rs3nWFdnIsEmGzWwLbfve/xyFFAizM8d1tYF5DxXWp+7f2c1Ssah+S
t+ua80N9Q2EwdZekQrtnfba58gW5RdTORNGAVjXJjQvHSytwkn1YlRsVQjhvv4Cy
R6LBb5Xdd0R9DIqu2mpp0bGvX6hlx0yPJrsiYxd1DsHl+aFdTnQ3OkZVvxe2MfWi
yhTIWQoLfHrMwc2l1qjn2c3x4AIRsqLiLkMTfgrgUWC+T2IL1oO5jFBjbeV3ljOY
fOfkGmG6TCsdF38qkB/fl869jUGXIBAHjQ==
</value>
</param>
</provider>
JWT Issuer
By default Knox only accept JWT with "iss": "KNOXSSO". Need to verify if IDP, for example Keycloak can customize it.
Alternatively, can override this in Knox configuration. Add following parameters in Ambari Knox Custom gateway-site
jwt.expected.issuer=CUSTOM_SSO jwt.expected.sigalg=RS256
However this new feature is only available in HDP3.x. Before upgrade to HDP3, have to hard code jwt issuer as "KNOXSSO".
User Account ID
Current Knox can only parse user account from jwt parameter "sub": "guest". If IDP, for example Ping Federation, uses "client_id", it would be a problem. And the value should be consistent with account id synchronized into Ranger. Normally it is uid in OpenLDAP.
JWT audiences
This is optional. It must meet what's configured in knox.token.audiences
Known Issues
Knox json parser issue
Before HDP2.6.5, Knox json parser doesn't support complex json raw format with multiple keys.
Knox treats it as invalid json format, and throw NullpointException.
Workaround JSON Issue
For HDP2.6.3, manually replace dependant JSON jar files.
SSH into every Knox node.
$ cd /usr/hdp/2.6.3.0-235/knox/dep
$ sudo wget http://repo1.maven.org/maven2/com/nimbusds/nimbus-jose-jwt/4.41.2/nimbus-jose-jwt-4.41.2.jar
$ sudo wget http://repo1.maven.org/maven2/com/jayway/jsonpath/json-path/2.4.0/json-path-2.4.0.jar
$ sudo wget http://repo1.maven.org/maven2/net/minidev/json-smart/2.3/json-smart-2.3.jar
$ sudo wget http://repo1.maven.org/maven2/net/minidev/asm/1.0.2/asm-1.0.2.jar
$ sudo mv json-path-0.9.1.jar json-path-0.9.1.jar.bak
$ sudo mv nimbus-jose-jwt-4.11.jar nimbus-jose-jwt-4.11.jar.bak
$ sudo mv json-smart-1.2.jar json-smart-1.2.jar.bak
Restart Knox from Ambari
