Member since
01-07-2019
217
Posts
135
Kudos Received
18
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1938 | 12-09-2021 09:57 PM | |
1867 | 10-15-2018 06:19 PM | |
9247 | 10-10-2018 07:03 PM | |
4033 | 07-24-2018 06:14 PM | |
1479 | 07-06-2018 06:19 PM |
02-02-2018
06:24 PM
Hi @Bob Thorman, It looks like you tried launching a cluster from an AMI that is supposed to only work with HDCloud. You must follow the instructions in the HDCloud documentation to make this work. Here is the latest doc link: https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.16.5/bk_hdcloud-aws/content/launch/index.html First you need to launch HDCloud and then you can use it to create clusters. Hope this helps.
... View more
12-05-2017
06:29 PM
@Saikrishna Tarapareddy You need to provide your AWS Access Key and Secret Key, assuming that you are authorized to access the bucket. http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html I have never used Nifi with S3 but my guess is that you wan to specify bucket name (not URL) and region like here https://community.hortonworks.com/articles/49467/integrating-apache-nifi-with-aws-s3-and-sqs.html, so what you did looks correct, just the AWS Access Key and Secret Key are missing.
... View more
08-16-2017
07:58 PM
3 Kudos
Cloudbreak is now available directly from the Azure Marketplace. The goal of this article is to help you get started: quickly launch Cloudbreak from the Azure Marketplace, log in to the Cloudbreak UI and create a Cloudbreak credential using the interactive method. Let's get started! Update This article was written in 2017. As of today, Azure Marketplace no longer includes the latest version of Cloudbreak. Fo the latest version use this quickstart. Overview We will be performing the following steps: Meet the Prerequisites. Launch Cloudbreak. Log in to the Cloudbreak UI. Create a Cloudbreak credential using the interactive method. Meet the Prerequisites 1.In order to launch Cloubdreak on the Azure Marketplace, you must have a Microsoft Azure account. If you don't have an account, you can set it up at https://azure.microsoft.com. 2.In order to provision clusters on Azure, Cloudbreak must be able to assume a sufficient Azure role ("Owner" or "Contributor") via Cloudbreak credential: Your account must have the "Owner" role in the subscription in order to create a Cloudbreak credential using the interactive credential method. Your account must have the "Contributor" role (or higher) in the subscription in order to create a Cloudbreak credential using the app-based credential method. To check the roles that your subscription has, in your Azure account navigate to Subscriptions. Launch Cloudbreak 1.Log in to your Azure Portal. 2.From the services menu, select Marketplace. 3.Search for “Cloudbreak” and click on “Cloudbreak for Hortonworks Data Platform”. 4.Click on the Create button under the listing to access the wizard. 5.Provide the following parameters on the Basics page of the wizard: Administrator email address: Enter your email address. You will use this email address to log in to the Cloudbreak web UI. Administrator user password and Confirm password: Enter a password. You will use this password to log in to the Cloudbreak web UI. VM Username: Enter your chosen username. If needed, you will use it to SSH to the VM on which Cloudbreak is running. VM SSH public key: Paste your public SSH key. You can copy your public SSH key to the clipboard using pbcopy < ~/.ssh/id_rsa.pub. If needed, you can generate a new SSH keypair using ssh-keygen -t rsa -b 4096 -C "your_email@example.com". Subscription: Select your existing subscription. Resource group: Azure uses resource groups to logically organize related resources. Create a new resource group for your Cloudbreak deployment. Location: Select the Azure region in which you would like to launch Cloudbreak. You can launch Cloudbreak and provision your clusters in all regions supported by Microsoft Azure. Clusters created via Cloudbreak can be in the same or different region as Cloudbreak; when you launch a cluster, you select the region in which to launch it. 6. Click OK to move to the next page in the wizard. 7.Provide the following parameters on the Advanced settings page of the wizard: Controller Instance Type: This is the VM type that will be used to launch Cloudbreak. The default 1x Standard DS2 v2 is sufficient. Allow connections to the cloud controller from this address range or default tag (e.g. Internet): Use the following tool to determine the CIDR based on your IP address: or you can enter “Internet” to allow connections from all. Virtual network: By default, a new virtual network called “cbdeployerVnet” is created. You can also select an existing network. Subnets: By default a subnet called “cbdeployerSubnet” is created. If you selected an existing network, you must configure your own subnet. 8. Click OK to move to the next page in the wizard. 9.On the Summary page of the wizard, confirm that all the information looks correct and click OK to proceed. 10.On the Buy page of the wizard, click on Purchase. This starts the launch process. 11.You will be redirected to your dashboard. On your dashboard and in the notifications, you will see that Cloudbreak is being deployed: Log in to the Cloudbreak UI 1.Once the deployment is complete, you will see “Deployments succeeded” in your notifications and a new tile with resources will be added to your dashboard: 2.Click on the tile to navigate to the resource group. 3.Click on Deployments. 4.Click on hortonworks.cloudbreak-for-hortonworks-data-platf-<>. 5. Copy the URL from Outputs > LOGINURL. 6. Paste the URL into your browser. 7.Your browser will warn you that the connection is unsafe. Proceed to the UI. 8. Log in with the administrator email address and password created in the first step of the Azure wizard. Create a Cloudbreak Credential 1.In the Cloudbreak web UI, click on +create credential. 2. The wizard will appear. Click Next. 3. On the second page of the credential wizard, provide a name for your credential. By default, an interactive credential is pre-selected, your public SSH key (which will be used to SSH to cluster VMs) is pre-entered, and Cloudbreak uses your existing “Contributor” role. If you would like to change any of these defaults, refer to the Cloudbreak on Azure Marketplace documentation. 4. Click on Next. 5. On the last page of the wizard, copy the code and click on Azure login. 6. You will be redirected to the Device Login page. Paste the code that you copied earlier. 7.Click Continue and then select your account. 8.Once you see the following page, you may close the window: 9. Go back to the Cloudbreak UI and make sure that the credential that you created is available and pre-selected at the top of the page: Congratulations! Now you can start creating clusters! For more information refer to the Cloudbreak on Azure Marketplace documentation.
... View more
Labels:
08-11-2017
06:17 PM
This is a new option available as TP in HDP 2.6.
... View more
07-28-2017
11:23 PM
If I am using Cloudbreak on GCP and trying to create a cluster with GCS file system: (Q1) Should I enter a name for an existing bucket or a name for a new bucket under Default Bucket Name? (Q2) How will this bucket be used by default?
... View more
Labels:
- Labels:
-
Hortonworks Cloudbreak
07-20-2017
09:48 PM
@cduby @DL Did you end up getting past this error and successfully creating a cluster?
... View more
06-28-2017
06:14 PM
Hi @raghavendra v, Did you set the following configuration properties to authenticate with S3? https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cloud-data-access/content/s3-config-props.html If copying from HDFS in a cluster on prem to S3, this is the setting that you need.
... View more
06-20-2017
05:13 PM
1 Kudo
@Phoncy Joseph To add to the previous answer: If you upgrade to HDP 2.6.1, there is a new Ranger feature that allows you to control access to S3 buckets via Hive by different user. It's a new "URL" parameter available when creating a Hive policy: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_security/content/hive_policy.html. So once you add multiple S3 buckets you can control which user can read or write from which buckets.
... View more
06-19-2017
07:32 PM
@bhavik shah 1. Before you upgraded, were you able to read and write from that S3 location? 2. In HDP 2.6.1, if you want to use Ranger to control access to a specific S3 location, you can use the URL parameter when defining a Hive policy: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_security/content/hive_policy.html Are you using that? If not, you could try to create a new Hive policy for the hive user to access the S3 location. This feature is technical preview, however.
... View more
06-16-2017
07:55 PM
2 Kudos
With HDCloud for AWS 1.16 and Cloudbreak 1.16.1, you can optionally use Hortonworks flex subscriptions to cover the controller and all clusters created. To do that you must:
Obtain a flex subscription. For general information about the Hortonworks Flex Support Subscription, visit the Hortonworks Support page at https://hortonworks.com/services/support/enterprise/. Configure SmartSenseID and Telemetry when launching the cloud controller (HDCloud for AWS) or Cloudbreak. Register your Flex Subscription in the cloud controller or Cloudbreak web UI. When creating a cluster, select a flex subscription to use for the cluster. Configure and Manage Flex in HDCloud 1. To configure flex in HDCloud, you must provide your SmartSense ID and Telemetry Opt In when launching the cloud controller. These options are available in the SmartSense Configuration section of the CloudFormation template: 2. Once you log in to the cloud controller web UI, you can manage your flex subscriptions from the Settings > Manage Flex Subscriptions: 3. From this page, you can: Register a new flex subscription. The first subscription that you add will be used as default for the cloud controller and all cluster created by it. Set a default flex subscription (if using multiple subscriptions). Select a flex subscription to be used for cloud controller (if using multiple subscriptions). Delete a flex subscription. Check which clusters are connected to a specific subscription. 4. When creating a cluster, you will see in the GENERAL CONFIGURATION > SHOW ADVANCED OPTIONS that your default flex subscription is automatically assigned to the cluster: 5. Equivalent options are also available via HDC CLI. See https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.16.0/bk_hdcloud-aws/content/cli-using/index.html#managing-flex-subscriptions. Configure and Manage Flex in Cloudbreak 1. To configure flex in Cloudbreak, enable SmartSense in the Profile by adding the following variables: export CB_SMARTSENSE_ID=YOUR-SMARTSENSE-ID
export CB_SMARTSENSE_CONFIGURE=true For example: export CB_SMARTSENSE_ID=A-00000000-C-00000000
export CB_SMARTSENSE_CONFIGURE=true You can do this in one of the two ways: - When initiating Cloudbreak - After you've already initiated Cloudbreak Deployer. In this case, you must restart Cloudbreak using cbd restart. 2. Once you log in to the Cloudbreak web UI, you can manage your flex subscriptions from the manage flex subscriptions tab: 3. From this page, you can:
Register a new flex subscription. Set a default flex subscription.
Select a flex subscription to be used for cloud controller.
Delete a flex subscription.
Check which clusters are connected to a specific subscription. 4. When creating a cluster using the advanced options, in the CONFIGURE CLUSTER > Flex Subscriptions, you can select the flex subscription that you want to use: 5. Equivalent options are also available via Cloudbreak Shell. Hortonworks Documentation https://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.16.0/bk_hdcloud-aws/content/flex/index.html http://sequenceiq.com/cloudbreak-docs/release-1.16.1/help/
... View more
Labels: