question Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller) in Archives of Support Questions (Read Only)

Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

snehasharma_85 — Sun, 18 Aug 2019 04:20:16 GMT

hi.. i'm new to Hadoop and AWS. I'm trying to create a cluster so that i can run Hadoop on that.

Everytime i'm getting the below error when i try to create the cluster. Can somebody please help me in understanding the issue.

Erorr from cloudbreak.log file is:(Attached the log)

cbreak_cloudbreak_1 | com.amazonaws.AmazonClientException: Unrecognized service response for the dry-run request. Caused by: com.amazonaws.services.ec2.model.AmazonEC2Exception: In order to use this AWS Marketplace product you need to accept terms and subscribe. To do so please visit http://aws.amazon.com/marketplace/pp?sku=1txpg2zpxhinmi3gpvauo19db (Service: AmazonEC2; Status Code: 401; Error Code: OptInRequired; Request ID: 1c06b485-761f-4eab-b595-ab88938c9500)

cbreak-log.txt

Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

rkovacs — Wed, 14 Jun 2017 15:52:08 GMT

Hi @Sneha Sharma, Could you please give me more details? Which version of HDC do you use? What type of cluster do you want to provision? A Cloudbreak logs would be nice from the controller pane located at /var/lib/cloudbreak-deployer/cbreak.log.

Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

snehasharma_85 — Wed, 14 Jun 2017 16:46:18 GMT

@rkovacs

HDC version is 2.6 , i got it from here - https://aws.amazon.com/marketplace/pp/B01M193KGR.

I wanted to have a small cluster just for learning purpose - with Hadoop, hive, Spark..etc, so i went for the trial version to see how it goes.

I have set the CIDR for all inbound traffic to 0.0.0.0/0 - all protocols in EC2. I'm able to access the HDC console - https://ec2-34-205-87-124.compute-1.amazonaws.com

Looks like i'm missing some configuration changes

Will send the logs in a short while.

Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

rkovacs — Wed, 14 Jun 2017 18:35:11 GMT

Hi @Sneha Sharma,

There was a bug around this one, and it was fixed in 1.16.0. Could you please terminate your existing stuff, and use latest Controller Service instead? https://aws.amazon.com/marketplace/pp/B01LXOQBOU. The easiest from here: https://aws.amazon.com/marketplace/fulfillment?productId=571fb43d-99f6-4182-8166-61c477473f09&launch=manualLaunch

Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

snehasharma_85 — Thu, 15 Jun 2017 17:58:05 GMT

hi @rkovacs,

Created the new controller service, using the link https://aws.amazon.com/marketplace/pp/B01LXOQBOU., but facing the same error 😞 Any help.

Will list down the steps that i did since day1:

1. created EC2 console, generated the key pair and deployed HDC

2. Created security groups in EC2 to control the inbound and outbound traffic to and from the control plane -- 0.0.0.0/0.

3. Logged into cloud controller console and tried to create a cluster(small cluster) by selecting the default options. Gave admin as username pwd for Ambari .

Am I missing something here ... 😞

Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

snehasharma_85 — Thu, 15 Jun 2017 22:19:10 GMT

Attached the error log.

Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

Dominika — Fri, 16 Jun 2017 01:52:44 GMT

Hi @Sneha Sharma

Did you subscribe to "Hortonworks Data Cloud - HDP Services"? Can you check your subscription status? You need to subscribe to two separate AWS Marketplace services (Controller Service and HDP Services) in order to run HDCloud.

The instructions for subscribing and checking subscription status are at https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.16.0/bk_hdcloud-aws/content/subscribe/index.html

Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

snehasharma_85 — Fri, 16 Jun 2017 12:18:54 GMT

Issue is resolved by subscribing to the HDP services. I wonder why this was not a mandatory item to select, given the fact that Cluster installation will fail if you don't subscribe to it. 😞

Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

snehasharma_85 — Fri, 16 Jun 2017 12:20:03 GMT

Thanks for replying @Dominika Bialek.. Its resolved now.

Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

Dominika — Fri, 16 Jun 2017 13:15:21 GMT

I'm glad it helped @Sneha Sharma. It's tricky to enforce this since AWS Marketplace is a separate UI.

Please accept the most helpful answer so we can close this thread.

Re: Unable to create a Cloudbreak cluster in AWS - using HDP 2.6(Hortonworks Cloud Controller)

rkovacs — Fri, 16 Jun 2017 14:35:03 GMT

"\"Hortonworks Data Cloud - HDP Services\" product is designed to be used in conjunction with the "Hortonworks Data Cloud - Controller Service"

"After you have subscribed to this product, you should also subscribe to the "Hortonworks Data Cloud - Controller Service" Marketplace product and use that product to first launch the cloud controller service."

source: https://aws.amazon.com/marketplace/pp/B01M193KGR

"To launch Hortonworks Data Cloud for AWS, you need to subscribe to the following Hortonworks Data Cloud products in AWS Marketplace:

source: https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.16.0/bk_hdcloud-aws/content/subscribe/index.html