Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

[#1]

hi.. i'm new to Hadoop and AWS. I'm trying to create a cluster so that i can run Hadoop on that.

Everytime i'm getting the below error when i try to create the cluster. Can somebody please help me in understanding the issue.

Erorr from cloudbreak.log file is:(Attached the log)

cbreak_cloudbreak_1 | com.amazonaws.AmazonClientException: Unrecognized service response for the dry-run request. Caused by: com.amazonaws.services.ec2.model.AmazonEC2Exception: In order to use this AWS Marketplace product you need to accept terms and subscribe. To do so please visit http://aws.amazon.com/marketplace/pp?sku=1txpg2zpxhinmi3gpvauo19db (Service: AmazonEC2; Status Code: 401; Error Code: OptInRequired; Request ID: 1c06b485-761f-4eab-b595-ab88938c9500)

cbreak-log.txt

[#2]

Hi @Sneha Sharma, Could you please give me more details? Which version of HDC do you use? What type of cluster do you want to provision? A Cloudbreak logs would be nice from the controller pane located at /var/lib/cloudbreak-deployer/cbreak.log.

[#3]

@rkovacs

HDC version is 2.6 , i got it from here - https://aws.amazon.com/marketplace/pp/B01M193KGR.

I wanted to have a small cluster just for learning purpose - with Hadoop, hive, Spark..etc, so i went for the trial version to see how it goes.

I have set the CIDR for all inbound traffic to 0.0.0.0/0 - all protocols in EC2. I'm able to access the HDC console - https://ec2-34-205-87-124.compute-1.amazonaws.com

Looks like i'm missing some configuration changes

Will send the logs in a short while.

[#4]

Attached the error log.

[#5]

Hi @Sneha Sharma,

There was a bug around this one, and it was fixed in 1.16.0. Could you please terminate your existing stuff, and use latest Controller Service instead? https://aws.amazon.com/marketplace/pp/B01LXOQBOU. The easiest from here: https://aws.amazon.com/marketplace/fulfillment?productId=571fb43d-99f6-4182-8166-61c477473f09&launch...

[#6]

hi @rkovacs,

Created the new controller service, using the link https://aws.amazon.com/marketplace/pp/B01LXOQBOU., but facing the same error 😞 Any help.

Will list down the steps that i did since day1:

1. created EC2 console, generated the key pair and deployed HDC

2. Created security groups in EC2 to control the inbound and outbound traffic to and from the control plane -- 0.0.0.0/0.

3. Logged into cloud controller console and tried to create a cluster(small cluster) by selecting the default options. Gave admin as username pwd for Ambari .

Am I missing something here ... 😞

[#7 · Accepted solution]

Hi @Sneha Sharma

Did you subscribe to "Hortonworks Data Cloud - HDP Services"? Can you check your subscription status? You need to subscribe to two separate AWS Marketplace services (Controller Service and HDP Services) in order to run HDCloud.

The instructions for subscribing and checking subscription status are at https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.16.0/bk_hdcloud-aws/content/subscr...

[#8]

Thanks for replying @Dominika Bialek.. Its resolved now.

[#9]

Issue is resolved by subscribing to the HDP services. I wonder why this was not a mandatory item to select, given the fact that Cluster installation will fail if you don't subscribe to it. 😞

[#10]

I'm glad it helped @Sneha Sharma. It's tricky to enforce this since AWS Marketplace is a separate UI.

Please accept the most helpful answer so we can close this thread.