How to create a datalake from an existing environment
Introduction
Continuing my previous article on creating a CDP AWS environment, this tutorial teaches you how to automate the creation of a datalake, including:
- Setting up proper roles and IDBroker mappings associated with your user and environment
- Creating a datalake
- Syncing users to FreeIPA
Here is the TL;DR: go to my github and run the scripts as instructed.
Automation scripts
Step 1: Create IAM and launch Data Lake
Create roles and mapping in your existing environment:
cdp_create_group_iam.sh <base_dir> <prefix> <region>
Create datalake:
cdp_create_datalake.sh <base_dir> <prefix>
Step 2: Verify periodically until datalake status is RUNNING
cdp_describe_dl.sh <prefix>
Step 3: Sync free IPA users
cdp_sync_users.sh <base_dir> <prefix>
Conclusion
This was a short and sweet tutorial, more fun to come playing with data lake clusters and experiences!
