How to create a datalake from an existing environment

[#1]

Introduction

Continuing my previous article on creating a CDP AWS environment, this tutorial teaches you how to automate the creation of a datalake, including:

• Setting up proper roles and IDBroker mappings associated with your user and environment
• Creating a datalake
• Syncing users to FreeIPA

Here is the TL;DR: go to my github and run the scripts as instructed.

Automation scripts

Step 1: Create IAM and launch Data Lake

Create roles and mapping in your existing environment:

cdp_create_group_iam.sh <base_dir> <prefix> <region> 

Create datalake:

cdp_create_datalake.sh <base_dir> <prefix> 

Step 2: Verify periodically until datalake status is RUNNING

cdp_describe_dl.sh <prefix> 

Step 3: Sync free IPA users

cdp_sync_users.sh <base_dir> <prefix> 

Conclusion

This was a short and sweet tutorial, more fun to come playing with data lake clusters and experiences!