Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cloudbreak & Atlas installation error when using blueprint

Highlighted

Re: Cloudbreak & Atlas installation error when using blueprint

Expert Contributor

@pdarvasi I know I haven't overwritten the default value because I can correctly install other blueprints afterwards without issue... how would one overwrite this value? Or how is this value calculated? Are there some specific logs I could look in to see what the value is being selected?

Here's the cloudbreak-shell script I used to create the cluster that reproduces the issue

blueprint create --name "blueprintImportIssue" --file DataScience.json
credential select --name cloudbreakcredential
blueprint select --name "blueprintImportIssue"
availabilityset create --name hadoop-pilot-as --platformFaultDomainCount TWO
instancegroup configure --AZURE --instanceGroup host_group_1 --nodecount 1 --templateName default-infrastructure-template-d4 --securityGroupName internal-ports-and-ssh --ambariServer false
instancegroup configure --AZURE --instanceGroup host_group_2 --nodecount 1 --templateName default-infrastructure-template-d4 --securityGroupName internal-ports-and-ssh --ambariServer false
instancegroup configure --AZURE --instanceGroup host_group_3 --nodecount 1 --templateName default-infrastructure-template-d4 --securityGroupName internal-ports-and-ssh --ambariServer false
instancegroup configure --AZURE --instanceGroup host_group_4 --nodecount 1 --templateName default-infrastructure-template-d4 --securityGroupName internal-ports-and-ssh --ambariServer false
instancegroup configure --AZURE --instanceGroup host_group_6 --nodecount 1 --templateName default-infrastructure-template --securityGroupName internal-ports-and-ssh --ambariServer true
instancegroup configure --AZURE --instanceGroup host_group_7 --nodecount 1 --templateName default-infrastructure-template --securityGroupName internal-ports-and-ssh --ambariServer false
instancegroup configure --AZURE --instanceGroup host_group_5 --nodecount 4 --templateName default-infrastructure-template-3drives --securityGroupName internal-ports-and-ssh --ambariServer false  --availabilitySetName hadoop-pilot-as
network select --name default-azure-network
stack create --AZURE --name hadoop-pilot-import-issue-rg  --region "Canada Central" --attachedStorageType PER_VM
cluster create --description "Haoop Pilot" --password password

Anything look fishy that might cause this overwrite that you are talking about. (I'm just waiting for the build to finish so I can run the command you asked me to run.)

I will afterward try and figure out how to use the --utilsRepoId command to see if that helps.

Highlighted

Re: Cloudbreak & Atlas installation error when using blueprint

Expert Contributor

@pdarvasi

I actually tried this but I'm clearly missing something:

cloudbreak-shell>cluster create --description "Haoop Pilot" --password password --enableKnoxGateway true --stackRepoId HDP-2.6 --stackBaseURL  http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.0.3 --stack HDP --utilsBaseURL  http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7 --utilsRepoId HDP-UTILS-1.1.0.21
Command failed java.lang.RuntimeException: may not be null
<br>
Highlighted

Re: Cloudbreak & Atlas installation error when using blueprint

Expert Contributor
cloudbreak-shell>script --file ecmake-cloudbreak-pilot-cluster.sh
credential select --name cloudbreakcredential
Credential selected, name: cloudbreakcredential
blueprint select --name "template-name"
Blueprint has been selected, name: template-name
availabilityset create --name hadoop-pilot-as --platformFaultDomainCount TWO
  Availability sets  name             faultDomainCount
  -----------------  ---------------  ----------------
  hadoop-pilot-as    hadoop-pilot-as  TWO


instancegroup configure --AZURE --instanceGroup host_group_1 --nodecount 1 --templateName default-infrastructure-template-d4 --securityGroupName internal-ports-and-ssh --ambariServer false 
  instanceGroup  templateId  nodeCount  type  securityGroupId  attributes
  -------------  ----------  ---------  ----  ---------------  ----------
  host_group_1   5           1          CORE  5                {}


instancegroup configure --AZURE --instanceGroup host_group_2 --nodecount 1 --templateName default-infrastructure-template-d4 --securityGroupName internal-ports-and-ssh --ambariServer false 
  instanceGroup  templateId  nodeCount  type  securityGroupId  attributes
  -------------  ----------  ---------  ----  ---------------  ----------
  host_group_2   5           1          CORE  5                {}
  host_group_1   5           1          CORE  5                {}


instancegroup configure --AZURE --instanceGroup host_group_3 --nodecount 1 --templateName default-infrastructure-template-d4 --securityGroupName internal-ports-and-ssh --ambariServer false 
  instanceGroup  templateId  nodeCount  type  securityGroupId  attributes
  -------------  ----------  ---------  ----  ---------------  ----------
  host_group_2   5           1          CORE  5                {}
  host_group_1   5           1          CORE  5                {}
  host_group_3   5           1          CORE  5                {}


instancegroup configure --AZURE --instanceGroup host_group_4 --nodecount 1 --templateName default-infrastructure-template-d4 --securityGroupName internal-ports-and-ssh --ambariServer false
  instanceGroup  templateId  nodeCount  type  securityGroupId  attributes
  -------------  ----------  ---------  ----  ---------------  ----------
  host_group_2   5           1          CORE  5                {}
  host_group_1   5           1          CORE  5                {}
  host_group_4   5           1          CORE  5                {}
  host_group_3   5           1          CORE  5                {}


instancegroup configure --AZURE --instanceGroup host_group_6 --nodecount 1 --templateName default-infrastructure-template --securityGroupName internal-ports-and-ssh --ambariServer true 
  instanceGroup  templateId  nodeCount  type     securityGroupId  attributes
  -------------  ----------  ---------  -------  ---------------  ----------
  host_group_2   5           1          CORE     5                {}
  host_group_1   5           1          CORE     5                {}
  host_group_4   5           1          CORE     5                {}
  host_group_3   5           1          CORE     5                {}
  host_group_6   4           1          GATEWAY  5                {}


instancegroup configure --AZURE --instanceGroup host_group_7 --nodecount 1 --templateName default-infrastructure-template --securityGroupName internal-ports-and-ssh --ambariServer false
  instanceGroup  templateId  nodeCount  type     securityGroupId  attributes
  -------------  ----------  ---------  -------  ---------------  ----------
  host_group_7   4           1          CORE     5                {}
  host_group_2   5           1          CORE     5                {}
  host_group_1   5           1          CORE     5                {}
  host_group_4   5           1          CORE     5                {}
  host_group_3   5           1          CORE     5                {}
  host_group_6   4           1          GATEWAY  5                {}


instancegroup configure --AZURE --instanceGroup host_group_5 --nodecount 4 --templateName default-infrastructure-template-3drives --securityGroupName internal-ports-and-ssh --ambariServer false  --availabilitySetName hadoop-pilot-as 
  instanceGroup  templateId  nodeCount  type     securityGroupId  attributes
  -------------  ----------  ---------  -------  ---------------  ------------------------------------------------------------
  host_group_7   4           1          CORE     5                {}
  host_group_2   5           1          CORE     5                {}
  host_group_1   5           1          CORE     5                {}
  host_group_4   5           1          CORE     5                {}
  host_group_3   5           1          CORE     5                {}
  host_group_6   4           1          GATEWAY  5                {}
  host_group_5   7           4          CORE     5                {availabilitySet={faultDomainCount=2, name=hadoop-pilot-as}}


network select --name default-azure-network
Network is selected with name: default-azure-network
stack create --AZURE --name hadoop-pilot-import-issue-rg  --region "Canada Central" --attachedStorageType PER_VM  
Stack creation started with id: '130' and name: 'hadoop-pilot-import-issue-rg'
cluster create --description "Haoop Pilot" --password password 
Cluster creation started
Script required 3.309 seconds to execute



then I do as you say... with some digging I got this... skip to the end... there is no entry in that table.

docker exec -it cbreak_commondb_1 psql -U postgres

show databases;

 \c cbdb


cbdb=# select * from clustercomponent where cluster_id = 130                                                                                                                                                ;
 id | componenttype | name | cluster_id | attributes 
----+---------------+------+------------+------------
(0 rows)
cbdb=# select * from cluster where name = 'hadoop-pilot-import-issue-rg'                                                                                                                                    ;
 id |  account   | creationfinished | creationstarted | description | emailneeded |             name             |                owner                 | secure |    status     |                                                                                                          statusreason                                                                                                           | upsince | blueprint_id |         username         |             password             |   ambariip    | stack_id | filesystem_id |              configstrategy              | ldaprequired | sssdconfig_id | enableshipyard | emailto | ldapconfig_id |                                                                                                                                                                                        attributes                                                                                                                                                                                        |     blueprintinputs      | cloudbreakambariuser |                 cloudbreakambaripassword                 | blueprintcustomproperties | kerberosconfig_id | topologyvalidation | customcontainerdefinition 
----+------------+------------------+-----------------+-------------+-------------+------------------------------+--------------------------------------+--------+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+--------------+--------------------------+----------------------------------+---------------+----------+---------------+------------------------------------------+--------------+---------------+----------------+---------+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------+----------------------+----------------------------------------------------------+---------------------------+-------------------+--------------------+---------------------------
 97 | seq1234567 |                  |   1506124194009 | Haoop Pilot | f           | hadoop-pilot-import-issue-rg | 8b8b515a-4092-44a6-9347-375c40fbd584 | f      | CREATE_FAILED | Cluster installation failed to complete, please check the Ambari UI for more details. You can try to reinstall the cluster with a different blueprint or fix the failures in Ambari and sync the cluster with Cloudbreak later. |         |         1360 | ITY78VsJUx3VoCqAPPrEGg== | 02qK4MHBrRbZXbzFtoBITR+vuFih29e/ | 52.233.43.237 |      130 |               | ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES | f            |               | f              |         |               | LZk3WV7IrzH4pRkiwQ+Cfj85eG/ZqxWC44tfRy54VuijBNiyPpjRaYMf3MYAc3BQaki5V10HOKLkiSZ1uaAUevQY4L4Hs67O7sFwB9DtGtVR5lUIhf+iVJEPaAGf/GMd5KPY0Zh0U6ldtPHpKFxzIcQWNSCfN+XBjOnzoOD/Jd/fxhBRVEmArcb2YdPC62y9yWt+cfwdNqPPw/ZFAv87YcG3J+1P80Su9v/D+L+fTPjY6bMU5R68U0l1xt8HN7pmb5YML31GSoNlAssrINMKNts4Zas8TbbSIwraRL+ljSFqY6fqE0/UkzQgIhfcR1vgIYRN6Y9oabNm9MnYHAvwhBG6G/rRi/Li2CYiRR8nxy/5Hjb/ytMMPA== | 4RSE6ooK4YB8uKcVYVVu7Q== | cloudbreak           | H/89H2rul1DhODpedIvc8tgkU7Of65YTjzqGGccQDCbJHUEo6Rt7pQ== | 9kh4l+L8qlywJkeJLbE4kg==  |                97 | f                  | {}
(1 row)


cbdb=# select * from clustercomponent where id = 97
cbdb-# ;
 id |    componenttype    |        name         | cluster_id |                                                                                                            attributes                                                                                                            
----+---------------------+---------------------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 97 | AMBARI_REPO_DETAILS | AMBARI_REPO_DETAILS |         16 | {"predefined":false,"version":"2.5.0.3-7","baseUrl":"http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.0.3","gpgKeyUrl":"http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins"}
(1 row)



So there is a cluster but there isn't an entry in the database for this.

Highlighted

Re: Cloudbreak & Atlas installation error when using blueprint

@Matt Andruff

This

cbdb=# select * from clustercomponent where id = 97

should be like this instead:

cbdb=# select * from clustercomponent where cluster_id = 97

There should be 3 rows.

And pls. write us the exact CB version as well! Thanks!

Highlighted

Re: Cloudbreak & Atlas installation error when using blueprint

Expert Contributor

@pdarvasi

Here is the corrected query

postgres=# \c cbdb
You are now connected to database "cbdb" as user "postgres".
cbdb=# select * from clustercomponent where cluster_id = 97
cbdb-# ;
 id  |      componenttype      |          name           | cluster_id |                                                                                                                                                                                                                    attributes                                                                                                                                                                                                                    
-----+-------------------------+-------------------------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 607 | AMBARI_REPO_DETAILS     | AMBARI_REPO_DETAILS     |         97 | {"predefined":false,"version":"2.5.0.3-7","baseUrl":"http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.5.0.3","gpgKeyUrl":"http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins"}
 608 | HDP_REPO_DETAILS        | HDP_REPO_DETAILS        |         97 | {"stack":{"repoid":"HDP-2.6","redhat6":"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.0.3","redhat7":"http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.0.3"},"util":{"repoid":"HDP-UTILS-1.1.0.21","redhat6":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6","redhat7":"http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7"},"verify":true,"hdpVersion":"2.6.0.3"}
 609 | AMBARI_DATABASE_DETAILS | AMBARI_DATABASE_DETAILS |         97 | {"vendor":"embedded","fancyName":"","name":"postgres","host":"localhost","port":5432,"userName":"ambari","password":"password"}
(3 rows)
<br>
Highlighted

Re: Cloudbreak & Atlas installation error when using blueprint

Expert Contributor

@pdarvasi cloudbreak-shell version: 1.16.4

Re: Cloudbreak & Atlas installation error when using blueprint

Expert Contributor

I just checked your blueprint file and the blueprint_name is missing in the blueprint config like this:

https://github.com/hortonworks/cloudbreak/blob/master/integration-test/src/main/resources/blueprint/...

Highlighted

Re: Cloudbreak & Atlas installation error when using blueprint

Expert Contributor

I was just be really honest about what was exported. I know it needs a name added to it and I do add a name to it. That is not part of the issue for this case. I should have called out that it was 'raw' and needed a name added.

Don't have an account?
Coming from Hortonworks? Activate your account here