Co-authored by Balazs Gaspar and Steffen Maerkl - more coming soon   The purpose of this article is to describe how to set up Keycloak as a SAML 2.0 Identity Provider for CDP Public Cloud, enabling any user in your organisation to use single sign-on to access CDP and its services, such as Cloudera Machine Learning or Cloudera Data Warehouse.   Keycloak is a popular open source Identity and Access Management (IAM) solution but of course there are many more such solutions, so see this as one example. In an upcoming Cloudera Community post we plan to provide a similar guide for configuring CDP Public Cloud with other popular SSO solutions.   In the examples below we will create users and groups directly in Keycloak. In a real world deployment it is more likely that you would use Keycloak only for SSO and rely on a separate provider (such as Active Directory) to store users and groups via LDAP federation. If you have not been exposed to SAML 2.0 previously, Okta has an excellent guide for beginners, including a diagram depicting the SAML authentication flow between the Identity Provider (IdP) and the Service Provider (SP).   One benefit of SAML is that it does not require the IdP and the SP to communicate directly to each other, the authentication is performed by a token exchange through your browser session. Therefore you can use any SSO solution that supports SAML 2.0, including cloud SSO services as well as local or remote open-source deployments.   Prerequisites: Make sure you familiarise yourself with the “Configuring identity providers” section of the CDP Public Cloud documentation You will need administrator privileges in the CDP Management Console to perform the actions in Step 2.2 Step 1 - Setting up Keycloak In the following we will present two methods: Download and run Keycloak server in standalone (no installation required, runs on Linux, Mac OS as well as Windows) Run in a Docker container Both methods enable you to run Keycloak on your localhost or a remote server (e.g. AWS EC2 or Azure VM).  Note: If you choose to deploy Keycloak on a remote server, make sure to set up and use a secure HTTPS connection to interact with the web interface (see SSL modes in the Keycloak documentation for details). Option A - Deploying Keycloak server locally Setting up Keycloak locally or on a cloud machine is straightforward and requires no installation (also no elevated permissions are required) - see the Keycloak server system requirements for details.   Briefly, these are the requirements to run the server in standalone mode: Keycloak can run on any operating system that runs Java Java 8 JDK zip (alternatively gzip and tar) At least 512M of RAM At least 1G of disk space To download and extract the Keycloak server binaries, we do the following:   wget https://downloads.jboss.org/keycloak/10.0.2/keycloak-10.0.2.zip unzip keycloak-10.0.2.zip KEYCLOAK_HOME=`pwd`/keycloak-10.0.2 PATH=$PATH:$KEYCLOAK_HOME/bin     While setting the KEYCLOAK_HOME path is not mandatory, it certainly makes working with the Keycloak command-line tools much easier.   We can now create an admin user and start the server in the background.   # Create an admin user add-user-keycloak.sh -r master -u admin -p admin # Start the keycloak server and verify it is running standalone.sh & curl http://localhost:8080   Opening http://localhost:8080 in a browser will allow you to log in with the admin user that you just created (admin/admin). By default, Keycloak will only accept unencrypted HTTP connections from the localhost and will enforce HTTPS for remote connections only (see SSL modes), which we also strongly recommend. Option B - Running Keycloak server in a container  Note: This option will not provide a persistent volume (unless you attach one) to store all the configs (realms, users, clients), so if you shut down the container your configuration will be lost.   Create an EC2 instance in AWS (we chose an Ubuntu 18.04) that meets the physical requirements listed in the Keycloak documentation (https://www.keycloak.org/docs/latest/server_installation/index.html) and make sure the Security Group you use allows incoming traffic on port 8443/22 to your IP address for the initial setup and configuration   SSH into the newly created EC2 instance and install the latest version of docker as described in the Docker documentation (https://docs.docker.com/install/linux/docker-ce/ubuntu/).   Bring up Keycloak by using the image (available in the public Docker Hub):   docker run -d -p 8443:8443 -e KEYCLOAK_USER=admin -e KEYCLOAK_PASSWORD=admin jboss/keycloak   Now that we have set up our Keycloak server, we can proceed in setting up our SAML Identity Provider. Step 2 - Setting up Identity Provider configuration (Realm) in Keycloak Log in to the Keycloak Administration Console: http://localhost:8080 (or https://<IP address of the Keycloak instance>:8443) Click on the Administration Console link and use admin/admin as the username and password to login.  Create a new realm as described in the documentation.  Step 2.1 - Generating the identity provider metadata Still in Keycloak, on the Realm Settings screen click on the “SAML 2.0 Identity Provider Metadata” link listed under “Endpoints”, which will open the descriptor file for the realm you created in the previous step. Copy the XML content. Step 2.2 - Setting up the identity provider in CDP Sign in to CDP, click on the Management Console tile followed by User Management on the left. Now navigate to the Identity Provider Tab and click on “Create Identity Provider”. After you provided a name, select “Direct Input” under “Provider Metadata”, paste the XML content from the previous setup into the input field, and click on “Create”. This adds the new identity provider to the list of CDP identity providers on the Identity Providers page. Also, note that CDP will generate a unique identifier and assign it to your Identity Provider. This ID will be added to our Keycloak configuration in the next step. Step 3 - Configuring your enterprise IdP to work with CDP as a service provider As a next step, we will add the CDP Control Plane as a SAML 2.0 client that uses our Keycloak deployment as an identity provider. See SAML clients in the Keycloak documentation for further details.   We have two ways to perform this: Importing the settings from a JSON configuration Setting up the SAML client manually Both methods rely on the CDP service provider configuration that is linked to the Identity Providers page (https://console.cdp.cloudera.com/iam/downloads/saml-metadata.xml). You will use this file if you choose Option B / manual setup. Option A - Automated client configuration To simplify the process, the authors of this article have created a sample Keycloak client configuration in JSON based on the metadata.xml that you just downloaded from CDP. This includes the required settings, SAML assertion mappings as well as signing certificates required by CDP.   Please note that this is by no means an “official” client configuration provided by Cloudera, but is rather meant to simplify the SAML client setup for experimenting or PoC environments. For everything else, You may want to use the metadata XML document that describes the CDP Service Provider (see Option B and the related CDP documentation) and build your own SAML client configuration.   { "clientId": "cdp-client", "surrogateAuthRequired": false, "enabled": true, "clientAuthenticatorType": "client-secret", "redirectUris": [ "https://consoleauth.altus.cloudera.com/saml" ], "webOrigins": [ "https://consoleauth.altus.cloudera.com" ], "notBefore": 0, "bearerOnly": false, "consentRequired": false, "standardFlowEnabled": true, "implicitFlowEnabled": false, "directAccessGrantsEnabled": false, "serviceAccountsEnabled": false, "publicClient": false, "frontchannelLogout": true, "protocol": "saml", "attributes": { "saml.assertion.signature": "true", "saml.force.post.binding": "true", "saml.multivalued.roles": "false", "saml.encrypt": "false", "saml_assertion_consumer_url_post": "https://consoleauth.altus.cloudera.com/saml?samlProviderId={ID}", "saml.server.signature": "true", "saml_idp_initiated_sso_url_name": "cdp-sso", "saml.server.signature.keyinfo.ext": "false", "exclude.session.state.from.auth.response": "false", "saml.signature.algorithm": "RSA_SHA256", "saml_force_name_id_format": "false", "saml.client.signature": "true", "tls.client.certificate.bound.access.tokens": "false", "saml.authnstatement": "true", "display.on.consent.screen": "false", "saml_name_id_format": "username", "saml.onetimeuse.condition": "false", "saml_signature_canonicalization_method": "http://www.w3.org/2001/10/xml-exc-c14n#" }, "authenticationFlowBindingOverrides": {}, "fullScopeAllowed": true, "nodeReRegistrationTimeout": -1, "protocolMappers": [ { "name": "my-email-id-mapper", "protocol": "saml", "protocolMapper": "saml-user-property-mapper", "consentRequired": false, "config": { "attribute.nameformat": "URI Reference", "user.attribute": "email", "friendly.name": "my-email-friendly-name", "attribute.name": "urn:oid:0.9.2342.19200300.100.1.3" } }, { "name": "my-groups-mapper", "protocol": "saml", "protocolMapper": "saml-group-membership-mapper", "consentRequired": false, "config": { "single": "true", "attribute.nameformat": "URI Reference", "full.path": "false", "friendly.name": "my-groups-friendly-name", "attribute.name": "https://altus.cloudera.com/SAML/Attributes/groups" } }, { "name": "my-firstname-mapper", "protocol": "saml", "protocolMapper": "saml-user-property-mapper", "consentRequired": false, "config": { "attribute.nameformat": "URI Reference", "user.attribute": "firstName", "friendly.name": "my-firstname-friendly-name", "attribute.name": "https://altus.cloudera.com/SAML/Attributes/firstName" } }, { "name": "my-lastname-mapper", "protocol": "saml", "protocolMapper": "saml-user-property-mapper", "consentRequired": false, "config": { "attribute.nameformat": "URI Reference", "user.attribute": "lastName", "friendly.name": "my-lastname-friendly-name", "attribute.name": "https://altus.cloudera.com/SAML/Attributes/lastName" } } ], "defaultClientScopes": [ "web-origins", "role_list", "profile", "roles", "email" ], "optionalClientScopes": [ "address", "phone", "offline_access", "microprofile-jwt" ], "access": { "view": true, "configure": true, "manage": true } }   To create a SAML 2.0 client in Keycloak via the above JSON, we need to click “Create” on the Clients page and Import the cdp-client.json file with the above content. The Client ID and Client Protocol fields will be filled in automatically, Client SAML Endpoint can be blank. Note that the four mappers we need are already configured here: "name": "email-mapper" → mandatory, this will be used to create the user name in CDP "name": "groups-mapper" → optional, asserts the user’s group memberships in a single concatenated attribute "name": "firstname-mapper” → optional, asserts the user’s first name "name": "lastname-mapper" → optional, asserts the user’s last name It is also worth noting that certain attributes in the configuration must be used exactly as in our example. CDP requires the four attributes to be named "urn:oid:0.9.2342.19200300.100.1.3" (or “mail” for short), "https://altus.cloudera.com/SAML/Attributes/groups", "https://altus.cloudera.com/SAML/Attributes/firstName" and "https://altus.cloudera.com/SAML/Attributes/lastName" (see the previously mentioned CDP documentation). When using Keycloak we also need to make sure to specify the proper user attributes “email”, “firstName” and “lastName”. Keycloak handles group membership assertion in a simplified way and will assert all groups of the user logging in via SSO. Other SSO providers offer a more sophisticated approach, enabling custom filtering or even group name transition during the SAML assertion. While on the page, take note of the URL displayed under “IDP Initiated SSO URL Name” in the client settings page. If you have set up Keycloak on your local machine, this should be “Target IDP initiated SSO URL: https://localhost:8443/auth/realms/cdp/protocol/saml/clients/cdp-sso” or similar. We will use this URL to perform SSO login to CDP in Step 4 of our guide.   To finalize the SAML client setup, click Edit in the Client list, and on the bottom of the client details page open the section “Fine Grain SAML Endpoint Configuration”. Notice that the value is set to “https://consoleauth.altus.cloudera.com/saml?samlProviderId={ID}”. Replace {ID} with the Identity Provider ID generated by CDP in the previous step and click Save. Option B - Manual client setup Create a new realm as described in the documentation.  Create a new Client by navigating to the Clients section and clicking on “Create”.  Import the saml-metadata.xml file that you download earlier from the CDP UI.   In the Client configuration, adjust the following parameters:   IDP Initiated SSO URL Name: samlclient (or any other arbitrary value)  (You can copy the “Target IDP initiated SSO URL” that’s being shown under this field and paste it into the “Base URL” field.) Assertion Consumer Service POST Binding URL” (you will need to expand the “Fine Grain SAML Endpoint Configuration” section): <”Single Sign On URL” from your IdP entry in CDP>  Last but not least you need to specify the Mappers. Click on the “Mappers” tab and add four entries for each of the attributes described in the saml-metadata.xml file of CDP (Email, First Name, Last Name, and Groups). You can find the link under the Identity Providers tab in CDP. You may note Email is the only mandatory attribute, whereas the First Name, Last Name, and Groups are optional. Under the Mappers tab, click on “Create” (one entry for each of the four attributes):   Email (mandatory):  Name: email-mapper (or any arbitrary name) Mapper Type: User Property Property: email Friendly Name: mail SAML Attribute Name: urn:oid:0.9.2342.19200300.100.1.3  SAML Attribute NameFormat: URI Reference   First Name (optional): Name: firstName (or any arbitrary name) Mapper Type: User Property Property: firstName Friendly Name: firstName SAML Attribute Name: https://altus.cloudera.com/SAML/Attributes/firstName SAML Attribute NameFormat: URI Reference   Last Name (optional):  Name: lastName (or any arbitrary name) Mapper Type: User Property Property: lastName Friendly Name: lastName SAML Attribute Name: https://altus.cloudera.com/SAML/Attributes/lastName  SAML Attribute NameFormat: URI Reference   Groups (optional):  Name: groups (or any arbitrary name) Mapper Type: Group list Group attribute name: https://altus.cloudera.com/SAML/Attributes/groups Friendly Name: groups SAML Attribute NameFormat: URI Reference Single Group Attribute: ON Full group path: OFF Step 4 - Adding users and groups So far you have set up your Keycloak IdP, configured it in the CDP management console, and appended the unique ID to the Assertion Consumer Service POST Binding URL. As a final step to test the deployment we only need one more thing - users!   You can follow the Keycloak documentation to create a group, a user, and add that user to the group (see https://www.keycloak.org/docs/latest/server_admin/#groups for details). When creating the user, the following fields should be filled in: Email First Name  Last Name “Username” is optional, as we will actually be passing the email attribute to CDP. Alternatively, you can also use “Username” instead of “Email”, just make sure to change the “Property: email” attribute for the “urn:oid:0.9.2342.19200300.100.1.3“ assertion in the previous step to “Property: username”. In addition to the above, a password also needs to be created under the “Credentials” tab.   An example user setup is shown below. Finally, we need to create a group (e.g “test-group”) and add our user to that group.   Now that we have a user with an email, first name, last name, and group membership, you can test how these are passed on to CDP. At the end of Step 3, we have located the URL for performing SSO authentication to access the CDP Management Console (this should be  https://localhost:8443/auth/realms/cdp/protocol/saml/clients/cdp-sso or similar). Now, with Keycloak there are two ways to authenticate: Open the link in a new browser window and enter the username and the password that you just created. First click “Impersonate” on the user details (or user list) screens. You will be redirected to the user’s account where you can change the password, set up two-factor authentication, and view the applications that you have access to. Notice that the client we have just set up is shown in the application list. Not we can open https://localhost:8443/auth/realms/cdp/protocol/saml/clients/cdp-sso in a new tab. Since you are already authenticated with the user, Keycloak will not ask for a password but will take you to the CDP Management Console landing page. Clicking in the bottom left corner and selecting “Profile” will take you to the CDP account page of your use, which should look like this.     Notice that all attributes (email, first name, last name) have been properly asserted. Moreover, on the Groups tab, you can see that CDP automatically created the “test-group” and added your user. In case you are not able to see certain details (and see an error “Please contact your admin.”), make sure that your user or your group has at least the “IamViewer” role assigned in CDP.   Congratulations, you have successfully set up single-sign-on access to CDP!    As a next step, you can set up fine-grain permissions in CDP Identity Management that are enforced by Cloudera SDX when working with data in CDP. If you would like to understand the details, we recommend reading with the CDP Security Overview documentation.        ... View more