Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Failed to Create HDP cluster in GCP using bdutil

Highlighted

Failed to Create HDP cluster in GCP using bdutil

New Contributor

I am creating a HDP cluster in Google Cloud Platform (GCP) using bdutil. But failed with error messages:

Command used:

./bdutil -e platforms/hdp/ambari_env.sh -P hdp -a deploy

Messages:

......................

Mon May 14 19:08:55 UTC 2018: Invoking on workers: ./ambari-setup.sh ..Mon May 14 19:08:55 UTC 2018: Invoking on master: ./ambari-setup.sh .Mon May 14 19:08:55 UTC 2018: Waiting on async 'ssh' jobs to finish. Might take a while... Mon May 14 19:09:40 UTC 2018: Exited 1 : gcloud --project=migration-test-200817 --quiet --verbosity=info compute ssh hdp-a-m --command=sudo su -l -c "cd ${PWD} && ./ambari-setup.sh" 2>>ambari-setup_deploy.stderr 1>>ambari-setup_deploy.stdou t --ssh-flag=-tt --ssh-flag=-oServerAliveInterval=60 --ssh-flag=-oServerAliveCou ntMax=3 --ssh-flag=-oConnectTimeout=30 --zone=us-east1-b Mon May 14 19:09:40 UTC 2018: Fetching on-VM logs from hdp-a-m INFO: Display format "default". Mon May 14 19:09:45 UTC 2018: Exited 1 : gcloud --project=migration-test-200817 --quiet --verbosity=info compute ssh hdp-a-w-1 --command=sudo su -l -c "cd ${PWD } && ./ambari-setup.sh" 2>>ambari-setup_deploy.stderr 1>>ambari-setup_deploy.std out --ssh-flag=-tt --ssh-flag=-oServerAliveInterval=60 --ssh-flag=-oServerAliveC ountMax=3 --ssh-flag=-oConnectTimeout=30 --zone=us-east1-b Mon May 14 19:09:45 UTC 2018: Fetching on-VM logs from hdp-a-w-1 Mon May 14 19:09:45 UTC 2018: Exited 1 : gcloud --project=migration-test-200817 --quiet --verbosity=info compute ssh hdp-a-w-0 --command=sudo su -l -c "cd ${PWD } && ./ambari-setup.sh" 2>>ambari-setup_deploy.stderr 1>>ambari-setup_deploy.std out --ssh-flag=-tt --ssh-flag=-oServerAliveInterval=60 --ssh-flag=-oServerAliveC ountMax=3 --ssh-flag=-oConnectTimeout=30 --zone=us-east1-b Mon May 14 19:09:45 UTC 2018: Fetching on-VM logs from hdp-a-w-0 INFO: Display format "default". INFO: Display format "default". Mon May 14 19:09:46 UTC 2018: Command failed: wait ${SUBPROC} on line 326. Mon May 14 19:09:46 UTC 2018: Exit code of failed command: 1 Mon May 14 19:09:46 UTC 2018: Detailed debug info available in file: /tmp/bdutil -20180514-190713-KRH/debuginfo.txt Mon May 14 19:09:46 UTC 2018: Check console output for error messages and/or ret ry your command.

Here are details in /tmp/bdutil-20180514-190713-KRH/debuginfo.txt

hdp-a-m: which: no apt-get in (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin)

hdp-a-m: mkdir: cannot create directory `/hadoop/hdfs': No such file or directory

Here are cluster I am setting up:

CONFIGBUCKET='mybucket'

PROJECT='myproject' GCE_IMAGE='' GCE_IMAGE_PROJECT='centos-cloud' GCE_IMAGE_FAMILY='centos-6' GCE_ZONE='us-east1-b' GCE_NETWORK='default' GCE_TAGS='bdutil' PREEMPTIBLE_FRACTION=0.0 PREFIX='hdp-a' NUM_WORKERS=2 MASTER_HOSTNAME='hdp-a-m' WORKERS='hdp-a-w-0 hdp-a-w-1' BDUTIL_GCS_STAGING_DIR='gs://tsy-hdp/bdutil-staging/hdp-a-m'

Thanks for any help.

1 REPLY 1

Re: Failed to Create HDP cluster in GCP using bdutil

New Contributor

For your reference, here is link to bdutill:

https://github.com/GoogleCloudPlatform/bdutil

Don't have an account?
Coming from Hortonworks? Activate your account here