Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error creating cluster on HortonWorks DataCloud

Highlighted

Error creating cluster on HortonWorks DataCloud

New Contributor

I've been trying to create a Data Science cluster (zeppelin 0.7, spark 2.1) using the hortonworks data cloud service on aws over the past few days but I keep getting the same error at the 'Starting Ambari cluster services' step. I've subscribed to both the hdc and hdp services and I launched the hdc in an existing vpc and subnet.

Timeout: Orchestrator component failed to finish in 7.500000 mins, last message: There are missing nodes from job (jid: 20171101165011520952), target: [ip-10-100-10-220.eu-west-1.compute.internal] Node: ip-10-100-10-220.eu-west-1.compute.internal Error(s): Error occurred installing package(s). Additional info follows: errors: - Loaded plugins: priorities, update-motd, upgrade-helper https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds') Trying other mirror. One of the configured repositories failed (GRAFANA), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" this: 1. Contact the upstream for the repository and get them to fix the problem. 2. Reconfigure the baseurl/etc. for the repository, to point to a working upstream. This is most often useful if you are using a newer distribution release than is supported by the repository (and the packages for the previous distribution release still work). 3. Disable the repository, so yum won't use it by default. Yum will then just ignore the repository until you permanently enable it again or use --enablerepo for temporary usage: yum-config-manager --disable Grafana 4. Configure the failing repository to be skipped, if it is unavailable. Note that yum will try to contact the repo. when it runs most commands, so will have to try and fail each time (and thus. yum will be be much slower). If it is a very temporary problem though, this is often a nice compromise: yum-config-manager --save --setopt=Grafana.skip_if_unavailable=true failure: repodata/repomd.xml from Grafana: [Errno 256] No more mirrors to try. https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds') https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds') https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds') https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds') https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds') https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds') https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds') https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds') https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds') https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on https://packagecloud.io/grafana/stable/el/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds')

After seeing the responses from this question I realised that my instance may not have internet access so I added outbound rules to allow all traffic on the instance's security group, however I still got the same error. After, I connected to the instance through ssh and pinged the packagecloud.io website that the cluster creation seems to be having issues connecting to, in order to verify that a connection was possible and it was able to connect. I'm creating the cluster in the same vpc and subnet as the hdc.

Today I tried again with larger instance sizes in the the cluster and the only difference was that the error was different. Any help is appreciated

Timeout: Orchestrator component failed to finish in 7.500000 mins, last message: Job: JobId{jobId='20171107101118277263'} is running currently
3 REPLIES 3

Re: Error creating cluster on HortonWorks DataCloud

Expert Contributor

Hi @George Hughes,

You mention two issues, but as i read first one is solved. Could you please share more details about the second one? For example which version do you use? Could you please attach log messages (only) for the cluster what failed? You can check what was going on by logging in to the gateway node and check salt logs under /var/log/salt*.

Re: Error creating cluster on HortonWorks DataCloud

New Contributor

Hi

I'm using version 1.16.2. I just tried this again as I deleted the stack from earlier and I got the first error again. I can't access the directory you mention as the nodes don't finish building so I can't access them through ssh

Re: Error creating cluster on HortonWorks DataCloud

Expert Contributor

Hi @George Hughes,

"Timeout" means node couldn't reach the repo, what looks like an environment issue not Cloudbreak one. The error occurred in the cluster creation phase not in stack provision, so the nodes must be there, and must be available via ssh at that time.