Custom parcels and CSDs allow partners and clients to bring their libraries and packages to CDP.  Traditionally, customers would use recipes to install parcels and CSDs or manually copy them to appropriate nodes. But, with CDP Public Cloud, you need a modular approach to do this. To address that need, Cloudera has developed the capability of specifying the parcel and CSD information as part of the cluster template (blueprint). With this approach, the parcel and CSD will get downloaded, distributed and activated as part of the cluster build process.   In the example below, FLINK 1.12 should be installed by default, but a different version of the FLINK parcel and CSD are passed for the Cloudera Manager to download, distribute and activate. FLINK service is added to the cluster during the cluster import. Steps to construct a custom cluster blueprint: Get the Image Catalog ID Get the AMI-ID of the DataLake. Log on the CDP > DataLake > Image Details (for a new/recently created environments). Create a temporary Datahub to get the AMI-ID, if the DL was created a while ago. Open the Image Catalog json Search for AMI-ID in the Image Catalog json Note the UUID. Make sure the image is in the right region and the cloud provider (AWS/Azure). Get the Cluster definition/Blueprint Log on to CDP > Environments > Cluster Definitions > Search for the appropriate version (depends on the DL). For this example, 7.2.2 - Streaming Analytics Light Duty for AWS  Click on the definition. Copy paste the raw json to an editor. Modify the Cluster definition Add environment name: This is where the cluster will be deployed.   "name": “DEFINITION_NAME”, "environmentName": “CDP_ENV_NAME”, "cluster": { "databases": [], "blueprintName": “7.2.2 - Streaming Analytics Light Duty with Apache Flink" "validateBlueprint": false,   Add CM base url, products: name, version and Parcels. Also, open Image catalog for the appropriate CM, CDH, Products (FLINK) versions and parcel location. CM:   "cm": { "enableAutoTls": true, "repository": { "baseUrl": "http://cloudera-build-us-west-1.vpc.cloudera.com/s3/build/4763198/cdh/7.x/parcels/", "version": “7.2.2” },   Products: CDH, FLINK   "products": [ { "name": "CDH", "version": "7.2.2-1.cdh7.2.2.p1.6575992", "parcel": "https://archive.cloudera.com/p/cdp-public/7.2.2.1/parcels/" }, { "name": "FLINK", "version": "1.10.0-csa1.2.1.0-cdh7.2.1.0-240-4844562", "parcel": "http://54.153.84.214/parcels/", "csd": [ "http://54.153.84.214/parcels/FLINK-1.10.0-csa1.2.1.0-cdh7.2.1.0-240-4844562.jar” ]}}   Image Catalog: Add image catalog at the end. Image ID:   "image": { "catalog": "cdp-default", "id": "eb1d458a-bf82-434b-bc2a-ac06b8cd6c0a"   Note: http://54.153.84.214/parcels is a temporary parcels location, this should point to the location of the parcel and CSDs. A finished custom blueprint should look like below:   { "instanceGroups": [ { "nodeCount": 1, "name": "manager", "type": "GATEWAY", "recoveryMode": "MANUAL", "template": { "aws": { "encryption": { "type": "NONE", "key": null } }, "instanceType": "m5.2xlarge", "rootVolume": { "size": 50 }, "attachedVolumes": [ { "size": 100, "count": 1, "type": "standard" } ], "cloudPlatform": "AWS" }, "recipeNames": [] }, { "nodeCount": 2, "name": "master", "type": "CORE", "recoveryMode": "MANUAL", "template": { "aws": { "encryption": { "type": "NONE", "key": null } }, "instanceType": "m5.2xlarge", "rootVolume": { "size": 50 }, "attachedVolumes": [ { "size": 100, "count": 1, "type": "standard" } ], "cloudPlatform": "AWS" }, "recipeNames": [] }, { "nodeCount": 3, "name": "worker", "type": "CORE", "recoveryMode": "MANUAL", "template": { "aws": { "encryption": { "type": "NONE", "key": null } }, "instanceType": "m5.2xlarge", "rootVolume": { "size": 50 }, "attachedVolumes": [ { "size": 100, "count": 1, "type": "standard" } ], "cloudPlatform": "AWS" }, "recipeNames": [] } ], "name": "pse-flink-cust", "environmentName": "pse-722-cdp-env", "cluster": { "databases": [], "exposedServices": [ "ALL" ], "blueprintName": "7.2.2 - Streaming Analytics Light Duty with Apache Flink", "validateBlueprint": false, "cm": { "enableAutoTls": true, "repository": { "baseUrl": "http://cloudera-build-us-west-1.vpc.cloudera.com/s3/build/4763198/cdh/7.x/parcels/", "version": "7.2.2" }, "products": [ { "name": "CDH", "version": "7.2.2-1.cdh7.2.2.p1.6575992", "parcel": "https://archive.cloudera.com/p/cdp-public/7.2.2.1/parcels/" }, { "name": "FLINK", "version": "1.10.0-csa1.2.1.0-cdh7.2.1.0-240-4844562", "parcel": "http://54.153.84.214/parcels/", "csd": [ "http://54.153.84.214/parcels/FLINK-1.10.0-csa1.2.1.0-cdh7.2.1.0-240-4844562.jar" ] } ] }, "Xcm": { "enableAutoTls": true, "repository": { "baseUrl": "https://archive.cloudera.com/p/cm-public/7.2.2-6458542/redhat7/yum/", "version": "7.2.2" }, "products": [ { "name": "CDH", "version": "7.2.2-1.cdh7.2.2.p1.6575992", "parcel": "https://archive.cloudera.com/p/cdp-public/7.2.2.1/parcels/" } ] } }, "image": { "catalog": "cdp-default", "id": "eb1d458a-bf82-434b-bc2a-ac06b8cd6c0a" }, "inputs": {} }   CDP CLI Provision to Cluster   $ cdp datahub --profile pse-demo create-aws-cluster --request-template file:///Users/njlamsal/flink-pse-demo.json --cluster-name protegrity-flink-1215     profile: Configure profile using cdpcli. request-template: Template created above. Note to use file:// cluster-name: Name of the cluster    ERROR: An error occurred: {"message":"Image or image catalog settings are incorrect: CloudbreakImageCatalogException: Inconsistent request, base images are disabled but custom repo information is submitted!"} (Status Code: 400; Error Code: INVALID_ARGUMENT; Service: datahub; Operation: createAWSCluster; Request ID: a7feb7a9-5547-43b2-8cfd-9543067677cc;)   Check DL and DH image IDs. If they are different, take the DH AMI-ID and find the catalog ID (UUID). ... View more