Hi @Dominika Bialek, few queries: 1. Can you please supply a custom blueprint for Data Science: Apache Spark 2, Apache Zeppelin/EDW-Analytics with HDP 3.1? 2. Also, if I use the custom blueprint (Data Science: Apache Spark 2, Apache Zeppelin with HDP3.0) supplied by you in this thread and try to create a 3 node cluster, there is a timeout happening due to which cluster creation is failing every time (this is not happening with HDP2.6 default blueprint). Operation timed out. Failed to find all '3' Ambari hosts. Stack: '34' 2/6/2019, 4:24:20 PM Building Ambari cluster; Ambari ip:172.31.90.36 2/6/2019, 4:04:06 PM Starting Ambari cluster services 2/6/2019, 4:02:12 PM Setting up infrastructure metadata 2/6/2019, 4:02:11 PM Bootstrapping infrastructure cluster 2/6/2019, 4:01:45 PM Infrastructure successfully provisioned 2/6/2019, 4:01:45 PM Snapshot of the error from the log: cloudbreak_1 | 2019-02-06 13:00:05,631 [reactorDispatcher-42] pollWithTimeout:56 INFO c.s.c.s.PollingService - [owner:d96fcce1-a991-4cf7-aa0b-6d186dc764bc] [type:CLUSTER] [id:34] [name:dip-test-cluster-hdp3x] [flow:84633730-dade-402b-b06b-8adf59f989a3] [tracking:53e509cd-6c8e-4c78-8d9f-2e79f6ec951e] Poller timeout. cloudbreak_1 | 2019-02-06 13:00:05,632 [reactorDispatcher-42] buildCluster:182 ERROR c.s.c.s.c.a.AmbariClusterSetupService - [owner:d96fcce1-a991-4cf7-aa0b-6d186dc764bc] [type:CLUSTER] [id:34] [name:dip-test-cluster-hdp3x] [flow:84633730-dade-402b-b06b-8adf59f989a3] [tracking:53e509cd-6c8e-4c78-8d9f-2e79f6ec951e] Error while building the Ambari cluster. Message Operation timed out. Failed to find all '3' Ambari hosts. Stack: '34', throwable: {} cloudbreak_1 | com.sequenceiq.cloudbreak.service.cluster.ambari.AmbariHostsUnavailableException: Operation timed out. Failed to find all '3' Ambari hosts. Stack: '34' Can you please advise? ... View more

Thanks @Dominika Bialek ... View more

@Dominika Bialek : Hello Dominika, another interesting thing I found out - if I change the stack version to 3.1 in your HDP3.0 blueprint to try to create HDP3.1 cluster, it fails with the following error (but runs well with 3.0) - Couldn't determine Ambari repo for the stack: <blueprint-name> Change made: "Blueprints":{ "blueprint_name":"hdp31-data-science-spark2-v4", "stack_name":"HDP", "stack_version":"3.1" }, So, does it mean the latest Cloudbreak version (2.7.x) can support HDP3.0 but not HDP3.1? Another follow up question: When you said "the plan was to only ship EDW-Analytics with HDP 3.x", does it mean the customization of blueprint will not be possible in the current Cloudbreak version to include other components that come with EDW-ETL? Please advise. ... View more

@pdarvasi: Ok, this is what my cb-policy.json looks like: { "Effect": "Allow", "Action": [ "iam:ListRolePolicies", "iam:GetRolePolicy", "iam:ListAttachedRolePolicies", "iam:ListInstanceProfiles", "iam:PutRolePolicy", "iam:PassRole", "iam:GetRole" ], "Resource": [ "*" ] }, { "Effect": "Allow", "Action": [ "iam:ListRolePolicies", "iam:GetRolePolicy", "iam:ListAttachedRolePolicies", "iam:ListInstanceProfiles", "iam:PutRolePolicy", "iam:PassRole", "iam:GetRole" ], "Resource": [ "*" ] }, { "Effect": "Allow", "Action": [ "autoscaling:CreateAutoScalingGroup", "autoscaling:CreateLaunchConfiguration", "autoscaling:DeleteAutoScalingGroup", "autoscaling:DeleteLaunchConfiguration", "autoscaling:DescribeAutoScalingGroups", "autoscaling:DescribeLaunchConfigurations", "autoscaling:DescribeScalingActivities", "autoscaling:DetachInstances", "autoscaling:ResumeProcesses", "autoscaling:SuspendProcesses", "autoscaling:UpdateAutoScalingGroup" ], "Resource": [ "*" ] }, { "Effect": "Allow", "Action": [ "kms:ListKeys", "kms:ListKeyPolicies", "kms:ListAliases" ], "Resource": "*" } ======================================================== Are you saying, I also need to add the below? { "Effect": "Allow", "Action": [ "ec2:*" ], "Resource": "*" } ... View more

@Dominika Bialek:Thanks for your quick response. ... View more

Hello Dominika, The blueprint in the example given was for Data Science: Apache Spark 2, Apache Zeppelin. Do you have a sample blueprint for HDP 3.0 - EDW-ETL: Apache Hive, Apache Spark 2 which I can run on Cloudbreak 2.7.x? ... View more