Member since
09-29-2015
94
Posts
37
Kudos Received
16
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1214 | 09-05-2018 02:30 PM | |
1321 | 08-28-2017 12:30 PM | |
1177 | 08-22-2017 09:18 AM | |
665 | 06-12-2017 07:22 PM | |
669 | 04-05-2017 07:06 PM |
01-07-2019
02:28 PM
@Ann Zacharia, Could you please explain the relation of this issue with Cloudbreak?
... View more
09-13-2018
11:11 AM
Hi, Do you see any Cloudbreak related image in your Openstack deployment?
... View more
09-07-2018
07:23 AM
What is the result of the following command on your Cloudbreak machine? curl -I http://public-repo-1.hortonworks.com/HDP/cloudbreak/cb-hdp-31-1807111345.img
... View more
09-06-2018
07:57 AM
To configure proxy settings for Cloudbreak Deployer, add the following configs to your Profile : export http_proxy="http://YOUR_PROXY_ADDRESS:YOUR_PROXY_PORT/"
export https_proxy="http(s)://YOUR_PROXY_ADDRESS:YOUR_PROXY_PORT/"
export CB_HTTP_PROXY="http://YOUR_PROXY_ADDRESS:YOUR_PROXY_PORT/"
export CB_HTTPS_PROXY="http(s)://YOUR_PROXY_ADDRESS:YOUR_PROXY_PORT/"
export CB_JAVA_OPTS="-Dhttp.proxyHost=YOUR_PROXY_ADDRESS -Dhttp.proxyPort=YOUR_PROXY_PORT -Dhttps.proxyHost=YOUR_PROXY_ADDRESS -Dhttps.proxyPort=YOUR_PROXY_PORT -Dhttp.nonProxyHosts=172.17.0.1|*.service.consul|*.node.dc1.consul"
... View more
09-05-2018
02:30 PM
1 Kudo
Hi @Rajkamal Mahamuni Natarajan, Here is an example how the instance profile needs to look like: "cloudStorage": {
"locations": null,
"s3": {
"instanceProfile": "arn:aws:iam::1234567890:instance-profile/full-access"
},
... View more
09-05-2018
01:07 PM
Seems the issue is the image is not available in your openstack installation. I don't know where are the official images are located, but you can burn your own any time. For more information please check https://github.com/hortonworks/cloudbreak-images
... View more
09-05-2018
12:58 PM
Could you please attach salt master and minion logs which located on the cluster node(s) at /var/log/salt directory. And also would be nice to check full stack trace copied from Cloudbreak log.
... View more
09-05-2018
12:56 PM
Hi @Marshal Tito, unfortunately there is no option from 2.0 to 2.7 (latest at that time) to organize roles and accounts. You have two options: Wait for the feature which is on the way Use 1.16.x until the feature arrives in version 2
... View more
09-05-2018
12:47 PM
This issue is not related to Cloudbreak. Could you please remove the tag?
... View more
09-05-2018
12:37 PM
Could you please explain what is your issue? Otherwise really hard to tell what went wrong.
... View more
06-19-2018
08:46 AM
Hi @William Prendergast, There is no way to solve your issue, because Cloudbreak uses some kind of round robin to balance nodes between multiple subnets. If you want to implement different subnet allocation strategies you have change the code here , and we are glad to merge pull requests, because your situation is make sense.
... View more
02-28-2018
07:18 AM
1 Kudo
We don't have. All of the operation are running in a reactive way in background. Cloudbreak has a non blocking async flow engine, which handle operations like upscale. But as i mentioned we have a subscriptions. You send a subscription request on that endpoint, and the system will send notifications about the state of the operations. Web UI works on the same way.
... View more
02-27-2018
10:03 PM
Hi @Sudhakara Vasiraju, stackendpoint.put() endpoint is just starts the upscale of the stack but it doesn't wait for the result. It happens asynchronously. There are no new ips at that time. So what you can do is: - poll get endpoint and check for new nodes in the instance groups - subscribe to evetns, and you will receive a message when Cloudbreak collected new metadata of the nodes on a websocket.
... View more
01-19-2018
02:24 PM
Be sure that is it possible to get disks on the same machine with the same account with google cloud's cli tool (gcloud)
... View more
01-19-2018
02:22 PM
Hi @Bhupesh Khanna, I tried to create a google credential on the mentioned version of Cloudbreak, and it works. There must be a network issue, or your credential information is not accepted by google, or your account doesn't have access to disks. Looks the issue is not on the Cloudbreak side, it asks google to retrieve available disks, and the connection timed out.
... View more
01-19-2018
12:28 PM
Hi, @Bhupesh Khanna, Seems Google api is not answering (at GcpCredentialConnector.listDisks()), could you please check internet connection on the running Cloudbreak instance? The easiest way is to execute `cbd doctor` command.
... View more
01-11-2018
10:39 AM
Only if the secret key i mean UAA_DEFAULT_SECRET in the Profile are the same on both installation. Not easy but not impossible.
... View more
01-09-2018
09:49 AM
1 Kudo
Hi @Airawat, Could you please file a Github issue at Cloudbreak project. The most important part is the steps how to reproduce the issue, what did you do, how was the custom network looks like. Thanks.
... View more
11-28-2017
04:20 PM
Hi @Nic Swart, Sorry for the late answer. Cloudbreak 1.16.5 has the environemnt variable CB_HOST_DISCOVERY_CUSTOM_DOMAIN.
... View more
11-08-2017
03:03 PM
Hi @George Hughes, "Timeout" means node couldn't reach the repo, what looks like an environment issue not Cloudbreak one. The error occurred in the cluster creation phase not in stack provision, so the nodes must be there, and must be available via ssh at that time.
... View more
11-07-2017
04:31 PM
Hi @George Hughes, You mention two issues, but as i read first one is solved. Could you please share more details about the second one? For example which version do you use? Could you please attach log messages (only) for the cluster what failed? You can check what was going on by logging in to the gateway node and check salt logs under /var/log/salt*.
... View more
11-06-2017
02:10 PM
Hi @Abhishek Sakhuja, This question is Ambari related not Cloudbreak, so what i can suggest to use more Ambari related tags on it, and hopefully Ambari experts should find it more easily.
... View more
10-31-2017
10:16 AM
Hi @Kumar Tg, It seems your cluster installation went fail. Does this correct? I just suggest because there is a "reinstall" button, which means Ambari is not configured well. That's why you can't see autoscaling tab.
... View more
08-28-2017
12:30 PM
1 Kudo
Hi @Julien Champ, So Cloudbreak doesn't configures that metric by default, but you should use Ambari API to enable it. Simply grab your JSON template from here, and POST it to Ambari on clusters/[cluster-name]/alert_definitions endpoint.
... View more
08-28-2017
07:06 AM
1 Kudo
Hi @Julien Champ, This feature was remove near 1.14 and unfortunately the documentation was not updated as well.
... View more
08-23-2017
01:09 PM
1 Kudo
Destination region of the cluster is the important one, and Cloudbreak requires preburned AMI there.
... View more
08-23-2017
12:49 PM
1 Kudo
Hi @Ramanathan Ramaiyah, Cloudbreak doesn't support that region. Unfortunately supporting a new region is not just adding new record into a "map", requires for example image burning and others. Should be a good idea to support Mumbai. Could you please create a feature request in github?
... View more
08-22-2017
09:47 AM
2 Kudos
Cloudbreak is a cloud agnostic Hadoop As A Service solution. It is easy to install, only one command in the command line, and an other one to start the service itself. During startup Cloudbreak Deployer tries to figure out the underlay infrastructure and sets required environment variables with competent default values, but some cases it isn't enough. In this article i collected all of the possible customizations, default values and some hint what does the variable configure in Profile file.
ADDRESS_RESOLVING_TIMEOUT 120000
- DNS lookup timeout for internal service discovery
AWS_ACCESS_KEY_ID ""
- Access key of the AWS account
AWS_ROLE_NAME cbreak-deployer
- Name of the AWS role for the `cbd aws [generate-rol, show role]` commands
AWS_SECRET_ACCESS_KEY ""
- Secret access key of the AWS account
AZURE_SUBSCRIPTION_ID
- Azure subscription ID for interactive login in Web UI
AZURE_TENANT_ID
- Azure tenant ID for interactive login in Web UI
CAPTURE_CRON_EXPRESSION
- SmartSense bundle generation time interval in Cron format
CBD_CERT_ROOT_PATH "${PWD}/certs"
- Path where deployer stores Cloudbreak certificates
CBD_LOG_NAME cbreak
- Name of the Cloudbreak log file
CBD_TRAEFIK_TLS "/certs/traefik/client.pem,/certs/traefik/client-key.pem"
- Path inside of the Traefik container where TLS files located
CB_AWS_CUSTOM_CF_TAGS ""
- Comma separated list of AWS CloudFormation Stack tags
CB_AWS_DEFAULT_CF_TAG ""
- Default tag AWS CloudFormation Stack
CB_AWS_DEFAULT_INBOUND_SECURITY_GROUP ""
- Default inbound policy name for AWS CloudFormation Stack
CB_AWS_EXTERNAL_ID provision-ambari
- External ID of the assume role policy
CB_AWS_HOSTKEY_VERIFY "false"
- Enables host fingerprint verification on AWS
CB_AWS_VPC ""
- Configures the VPC id on AWS if it is the same as provisioned cluster
CB_BLUEPRINT_DEFAULTS "hdp-small-default;hdp-spark-cluster;hdp-streaming-cluster"
- Comma separated list of the default blueprints what Cloudbreak initialize in database
CB_BYOS_DFS_DATA_DIR "/hadoop/hdfs/data"
- Deprecated - Default data dir for BYOP orchestrators
CB_COMPONENT_CLUSTER_ID
- SmartSense component cluster ID
CB_COMPONENT_ID
- SmartSense component ID
CB_COMPOSE_PROJECT cbreak
- Name of the Docker Compose project, will appear in container names too
CB_DB_ENV_DB "cbdb"
- Name of the Cloudbreak database
CB_DB_ENV_PASS ""
- Password for the Cloudbreak database authentication
CB_DB_ENV_SCHEMA "public"
- Used schema in the Cloudbreak database
CB_DB_ENV_USER "postgres"
- User for the Cloudbreak database authentication
CB_DB_ROOT_PATH "/var/lib/cloudbreak"
- Deprecated - Location of the database volume on Cloudbreak host
CB_DEFAULT_SUBSCRIPTION_ADDRESS http://xxxx.xxxx.xxxx:3000/notifications
- Address of the default subscription for Cloudbreak notifications
CB_ENABLEDPLATFORMS
- Disables Cloudbreak resource called Platform
CB_ENABLE_CUSTOM_IMAGE "false"
- Flag to enable custom cloud images
CBD_FORCE_START
- Disables docker-compose.yml and uaa.yml validation
CB_GCP_HOSTKEY_VERIFY "false"
- Enables host fingerprint verification on GCP
CB_HBM2DDL_STRATEGY "validate"
- Configures hibernate.xxxxxxx.auto in Cloudbreak
CB_HOST_DISCOVERY_CUSTOM_DOMAIN ""
- Custom domain of the provisioned cluster
CB_HTTPS_PROXY ""
- HTTPS proxy url
CB_HTTP_PROXY ""
- HTTP proxy url
CB_IMAGE_CATALOG_URL "https://name.amazonaws.com/cloudbreak-info/cb-image-catalog.json"
- Image catalog url
CB_INSTANCE_NODE_ID
- Unique identifier of the Cloudbreak node
CB_INSTANCE_PROVIDER
- Cloud provider of the Cloudbreak instance
CB_INSTANCE_REGION
- Cloud region of the Cloudbreak instance
CB_INSTANCE_UUID
- Unique identifier of Cloudbreak deployment
CB_JAVA_OPTS ""
- Extra Java options for Autoscale and Cloudbreak
CB_LOG_LEVEL "INFO"
- Log level of the Cloudbreak service
CB_MAX_SALT_NEW_SERVICE_RETRY 90
- Salt orchestrator max retry count
CB_MAX_SALT_RECIPE_EXECUTION_RETRY 90
- Salt orchestrator max retry count for recipes
CB_PLATFORM_DEFAULT_REGIONS
- Comma separated list of default regions by platform (AWS:eu-west-1)
CB_PRODUCT_ID
- SmartSense product ID
CB_SCHEMA_MIGRATION_AUTO true
- Flag for Cloudbreak automatic database schema update
CB_SMARTSENSE_CONFIGURE "false"
- Flag to install and configure SmartSense on cluster nodes
CB_SMARTSENSE_CLUSTER_NAME_PREFIX
- SmartSense Cloudbreak cluster name prefix
CB_SMARTSENSE_ID ""
- SmartSense subscription ID
CB_TEMPLATE_DEFAULTS "minviable-gcp,minviable-azure,minviable-aws"
- Comma separated list of the default templates what Cloudbreak initialize in database
CB_UI_MAX_WAIT 400
- Wait timeout for `cbd start-wait` command
CERTS_BUCKET ""
- S3 bucket name for backup and restore certificates via `cbd aws [certs-restore-s3 certs-upload-s3]` commands
CERT_VALIDATION "true"
- Enables cert validation in Cloudbreak and Autoscale
CLOUDBREAK_SMTP_AUTH "true"
- Configures mail.smtp.auth in Cloudbreak
CLOUDBREAK_SMTP_SENDER_FROM "noreply@hortonworks.com"
- Email address of the sender
CLOUDBREAK_SMTP_SENDER_HOST "smtp.service.consul"
- SMTP server address ot hostname
CLOUDBREAK_SMTP_SENDER_PASSWORD "$LOCAL_SMTP_PASSWORD"
- Password
CLOUDBREAK_SMTP_SENDER_PORT 25
- Port of the SMTP server
CLOUDBREAK_SMTP_SENDER_USERNAME "admin"
- User name for SMTP authentication
CLOUDBREAK_SMTP_STARTTLS_ENABLE "false"
- Configures mail.smtp.starttls.enable in Cloudbreak
CLOUDBREAK_SMTP_TYPE "smtp"
- Defines mail.transport.protocol in CLoudbreak
COMMON_DB commondb
- Name of the database container
COMMON_DB_VOL common
- Name of the database volume
COMPOSE_HTTP_TIMEOUT 120
- Docker Compose execution timeout
DB_DUMP_VOLUME cbreak_dump
- Name of the database dump volume
DB_MIGRATION_LOG "db_migration.log"
- Database migration log file
DOCKER_CONSUL_OPTIONS ""
- Extra options for Consul
DOCKER_IMAGE_CBD_SMARTSENSE hortonworks/cbd-smartsense
- SmartSense Docker image name
DOCKER_IMAGE_CLOUDBREAK hortonworks/cloudbreak
- Cloudbreak Docker image name
DOCKER_IMAGE_CLOUDBREAK_AUTH hortonworks/cloudbreak-auth
- Authentication service Docker image name
DOCKER_IMAGE_CLOUDBREAK_PERISCOPE hortonworks/cloudbreak-autoscale
- Autoscale Docker image name
DOCKER_IMAGE_CLOUDBREAK_SHELL hortonworks/cloudbreak-shell
- Cloudbreak Shell Docker image name
DOCKER_IMAGE_CLOUDBREAK_WEB hortonworks/cloudbreak-web
- Web UI Docker image name
DOCKER_TAG_ALPINE 3.1
- Alpine container version
DOCKER_TAG_CBD_SMARTSENSE 0.10.0
- SmartSense container version
DOCKER_TAG_CERT_TOOL 0.2.0
- Cert tool container version
DOCKER_TAG_CLOUDBREAK 2.1.0-dev.70
- Cloudbreak container version
DOCKER_TAG_CLOUDBREAK_SHELL 2.1.0-dev.70
- Cloudbreak Shell container version
DOCKER_TAG_CONSUL 0.5
- Consul container version
DOCKER_TAG_HAVEGED 1.1.0
- Haveged container version
DOCKER_TAG_MIGRATION 1.0.0
- Migration container version
DOCKER_TAG_PERISCOPE 2.1.0-dev.70
- Autoscale container version
DOCKER_TAG_POSTFIX latest
- Postfix container version
DOCKER_TAG_POSTGRES 9.6.1-alpine
- Postgresql container version
DOCKER_TAG_REGISTRATOR v5
- Registrator container version
DOCKER_TAG_SULTANS 2.1.0-dev.70
- Authentication service container version
DOCKER_TAG_TRAEFIK v1.2.0
- Traefik container version
DOCKER_TAG_UAA 3.6.5
- Identity container version
DOCKER_TAG_ULUWATU 2.1.0-dev.70
- Web UI container version
IDENTITY_DB_NAME "uaadb"
- Name of the Identity database
IDENTITY_DB_PASS ""
- Password for the Identity database authentication
IDENTITY_DB_URL "${COMMON_DB}.service.consul:5432"
- Url for the Identity database connection included the port number
IDENTITY_DB_USER "postgres"
- User for the Identity database authentication
LOCAL_SMTP_PASSWORD "$UAA_DEFAULT_USER_PW"
- Default password for the internal mail server
PERISCOPE_DB_HBM2DDL_STRATEGY "validate"
- Configures hibernate.hbm2ddl.auto in Autoscale
PERISCOPE_DB_NAME "periscopedb"
- Name of the Autoscale database
PERISCOPE_DB_PASS ""
- Password for the Autoscale database authentication
PERISCOPE_DB_SCHEMA_NAME "public"
- Used schema in the Autoscale database
PERISCOPE_DB_USER "postgres"
- User for the Autoscale database authentication
PERISCOPE_DB_TCP_ADDR
- Address of the Autoscale database
PERISCOPE_DB_TCP_PORT
- Port number of the Autoscale database
PERISCOPE_LOG_LEVEL "INFO"
- Log level of the Autoscale service
PERISCOPE_SCHEMA_MIGRATION_AUTO true
- Flag for Autoscale automatic database schema update
PUBLIC_IP
- Ip address or hostname of the public interface
REST_DEBUG "false"
- Enables REST call debug level in Cloudbreak and Autoscale
SL_ADDRESS_RESOLVING_TIMEOUT
- DNS lookup timeout of Authentication service for internal service discovery
SL_NODE_TLS_REJECT_UNAUTHORIZED "0"
- Enables self signed certifications in Authentication service
SULTANS_CONTAINER_PATH /sultans
- Default project location in Authentication service container
TRAEFIK_MAX_IDLE_CONNECTION 100
- Configures --maxidleconnsperhost for Traefik
UAA_CLOUDBREAK_ID cloudbreak
- Identity of the Cloudbreak scope in Identity
UAA_CLOUDBREAK_SECRET $UAA_DEFAULT_SECRET
- Secret of the Cloudbreak scope in Identity
UAA_CLOUDBREAK_SHELL_ID cloudbreak_shell
- Identity of the Cloudbreak Shell scope in Identity
UAA_DEFAULT_ACCOUNT "seq1234567.SequenceIQ"
- Default account for users as an Identity group
UAA_DEFAULT_SECRET
- Default secret for all the scopes and encryptions
UAA_DEFAULT_USER_EMAIL admin@example.com
- Email address of default admin user
UAA_DEFAULT_USER_FIRSTNAME Joe
- First name of default admin user
UAA_DEFAULT_USER_GROUPS "openid,cloudbreak.networks,cloudbreak.securitygroups,cloudbreak.templates,cloudbreak.blueprints,cloudbreak.credentials,cloudbreak.stacks,sequenceiq.cloudbreak.admin,sequenceiq.cloudbreak.user,sequenceiq.account.${UAA_DEFAULT_ACCOUNT},cloudbreak.events,cloudbreak.usages.global,cloudbreak.usages.account,cloudbreak.usages.user,periscope.cluster,cloudbreak.recipes,cloudbreak.blueprints.read,cloudbreak.templates.read,cloudbreak.credentials.read,cloudbreak.recipes.read,cloudbreak.networks.read,cloudbreak.securitygroups.read,cloudbreak.stacks.read,cloudbreak.sssdconfigs,cloudbreak.sssdconfigs.read,cloudbreak.platforms,cloudbreak.platforms.read"
- Default user groups of the users
UAA_DEFAULT_USER_LASTNAME Admin
- Last name of default admin user
UAA_DEFAULT_USER_PW
- Password of default admin user
UAA_FLEX_USAGE_CLIENT_ID flex_usage_client
- Identity of the Flex usage generator scope in Identity
UAA_FLEX_USAGE_CLIENT_SECRET $UAA_DEFAULT_SECRET
- Secret of the Flex usage generator scope in Identity
UAA_PERISCOPE_ID periscope
- Identity of the Autoscale scope in Identity
UAA_PERISCOPE_SECRET $UAA_DEFAULT_SECRET
- Secret of the Autoscale scope in Identity
UAA_PORT 8089
- Identity service public port
UAA_SULTANS_ID sultans
- Identity of the Authentication service scope in Identity
UAA_SULTANS_SECRET $UAA_DEFAULT_SECRET
- Secret of the Authentication service scope in Identity
UAA_ULUWATU_ID uluwatu
- Identity of the Web UI scope in Identity
UAA_ULUWATU_SECRET $UAA_DEFAULT_SECRET
- Secret of the Web UI scope in Identity
UAA_ZONE_DOMAIN example.com
- External domain name for zone in Identity
ULUWATU_CONTAINER_PATH /xxxxx
- Default project location in Web UI container
ULU_DEFAULT_SSH_KEY ""
- Default SSH key for the credentials in Cloudbreak
ULU_HOST_ADDRESS "https://$PUBLIC_IP"
- Web UI host
ULU_NODE_TLS_REJECT_UNAUTHORIZED "0"
- Enables self signed certifications in Web UI
ULU_OAUTH_REDIRECT_URI "$ULU_HOST_ADDRESS/authorize"
- Authorization page on Web UI
ULU_SUBSCRIBE_TO_NOTIFICATIONS "false"
- Flag for automatic subscriptions for CLoudbreak events
ULU_SULTANS_ADDRESS "https://$PUBLIC_IP/sl"
- Authentication service address
VERBOSE_MIGRATION false
- Flag of verbose database migration
There are some local Cloudbreak local development related variables.
CB_LOCAL_DEV_BIND_ADDR "192.168.59.3"
- Ambassador external address for local development of Cloudbreak and Autoscale
CB_SCHEMA_SCRIPTS_LOCATION "container"
- Location of Cloudbreak schema update files
DOCKER_TAG_AMBASSADOR 0.5.0
- Ambassador container version for local development
PERISCOPE_SCHEMA_SCRIPTS_LOCATION "container"
- Location of Cloudbreak schema update files
PRIVATE_IP $BRIDGE_IP
- Ip address or hostname of the private interface
REMOVE_CONTAINER "--rm"
- Keeps side effect containers for debug purpose
SULTANS_VOLUME_HOST /dev/null
- Location of the locally developed Authentication service project
UAA_SCHEMA_SCRIPTS_LOCATION "container"
- Location of Identity schema update files
ULUWATU_VOLUME_HOST /dev/null
- Location of the locally developed Web UI project
And also some for run Cloudbreak on macOS.
DOCKER_MACHINE ""
- Name of the Docker Machine where Cloudbreak runs
DOCKER_PROFILE Profile
- Profile file for Docker Machine related environment variables
MACHINE_CPU 2
- Number of the CPU cores on the Docker Machine instance
MACHINE_MEM 4096
- Amount of RAM on the Docker Machine instance
MACHINE_NAME cbd
- Name of the Docker Machine instance
MACHINE_OPTS "--xhyve-virtio-9p"
- Extra options for Docker Machine instance
MACHINE_STORAGE_PATH $HOME/.docker/machine
- Docker Machine storage path
... View more
- Find more articles tagged with:
- Cloud & Operations
- Cloudbreak
- FAQ
Labels:
08-22-2017
09:18 AM
Hi @Shyam Shaw, You are right, 1.16.1 doesn't contain the advanced options, but you can use the Cloudbreak wide environment variable. Unfortunately it will apply domain name for all the clusters what you spin up.
... View more
08-22-2017
06:24 AM
1 Kudo
Duplicated entry: https://community.hortonworks.com/questions/130799/specify-rds-for-hive-metastore-in-cloudbreak.html?childToView=130819
... View more