Support Questions
Find answers, ask questions, and share your expertise

Failed to Create HDP cluster in GCP using bdutil

Explorer

I am creating a HDP cluster in Google Cloud Platform (GCP) using bdutil. But failed with error messages:

Command used:

./bdutil -e platforms/hdp/ambari_env.sh -P hdp -a deploy

Messages:

......................

Mon May 14 19:08:55 UTC 2018: Invoking on workers: ./ambari-setup.sh ..Mon May 14 19:08:55 UTC 2018: Invoking on master: ./ambari-setup.sh .Mon May 14 19:08:55 UTC 2018: Waiting on async 'ssh' jobs to finish. Might take a while... Mon May 14 19:09:40 UTC 2018: Exited 1 : gcloud --project=migration-test-200817 --quiet --verbosity=info compute ssh hdp-a-m --command=sudo su -l -c "cd ${PWD} && ./ambari-setup.sh" 2>>ambari-setup_deploy.stderr 1>>ambari-setup_deploy.stdou t --ssh-flag=-tt --ssh-flag=-oServerAliveInterval=60 --ssh-flag=-oServerAliveCou ntMax=3 --ssh-flag=-oConnectTimeout=30 --zone=us-east1-b Mon May 14 19:09:40 UTC 2018: Fetching on-VM logs from hdp-a-m INFO: Display format "default". Mon May 14 19:09:45 UTC 2018: Exited 1 : gcloud --project=migration-test-200817 --quiet --verbosity=info compute ssh hdp-a-w-1 --command=sudo su -l -c "cd ${PWD } && ./ambari-setup.sh" 2>>ambari-setup_deploy.stderr 1>>ambari-setup_deploy.std out --ssh-flag=-tt --ssh-flag=-oServerAliveInterval=60 --ssh-flag=-oServerAliveC ountMax=3 --ssh-flag=-oConnectTimeout=30 --zone=us-east1-b Mon May 14 19:09:45 UTC 2018: Fetching on-VM logs from hdp-a-w-1 Mon May 14 19:09:45 UTC 2018: Exited 1 : gcloud --project=migration-test-200817 --quiet --verbosity=info compute ssh hdp-a-w-0 --command=sudo su -l -c "cd ${PWD } && ./ambari-setup.sh" 2>>ambari-setup_deploy.stderr 1>>ambari-setup_deploy.std out --ssh-flag=-tt --ssh-flag=-oServerAliveInterval=60 --ssh-flag=-oServerAliveC ountMax=3 --ssh-flag=-oConnectTimeout=30 --zone=us-east1-b Mon May 14 19:09:45 UTC 2018: Fetching on-VM logs from hdp-a-w-0 INFO: Display format "default". INFO: Display format "default". Mon May 14 19:09:46 UTC 2018: Command failed: wait ${SUBPROC} on line 326. Mon May 14 19:09:46 UTC 2018: Exit code of failed command: 1 Mon May 14 19:09:46 UTC 2018: Detailed debug info available in file: /tmp/bdutil -20180514-190713-KRH/debuginfo.txt Mon May 14 19:09:46 UTC 2018: Check console output for error messages and/or ret ry your command.

Here are details in /tmp/bdutil-20180514-190713-KRH/debuginfo.txt

hdp-a-m: which: no apt-get in (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin)

hdp-a-m: mkdir: cannot create directory `/hadoop/hdfs': No such file or directory

Here are cluster I am setting up:

CONFIGBUCKET='mybucket'

PROJECT='myproject' GCE_IMAGE='' GCE_IMAGE_PROJECT='centos-cloud' GCE_IMAGE_FAMILY='centos-6' GCE_ZONE='us-east1-b' GCE_NETWORK='default' GCE_TAGS='bdutil' PREEMPTIBLE_FRACTION=0.0 PREFIX='hdp-a' NUM_WORKERS=2 MASTER_HOSTNAME='hdp-a-m' WORKERS='hdp-a-w-0 hdp-a-w-1' BDUTIL_GCS_STAGING_DIR='gs://tsy-hdp/bdutil-staging/hdp-a-m'

Thanks for any help.

1 REPLY 1

Explorer

For your reference, here is link to bdutill:

https://github.com/GoogleCloudPlatform/bdutil

; ;