EC2 hosts unreachable

[#1]

I'm getting the following error when running ansible-playbook -i ec2.py playbook.yml.

TASK [Sanity check Metron web] *************************************************
fatal: [localhost -> localhost]: FAILED! => {"changed": false, "elapsed": 20, "failed": true, "msg": "Timeout when waiting for ec2-52-39-145-200.us-west-2.compute.amazonaws.com:5000"}
	to retry, use: --limit @playbook.retry


PLAY RECAP *********************************************************************
ec2-52-26-145-189.us-west-2.compute.amazonaws.com : ok=2    changed=0    unreachable=1    failed=0
ec2-52-33-71-234.us-west-2.compute.amazonaws.com : ok=2    changed=0    unreachable=1    failed=0
ec2-52-34-108-164.us-west-2.compute.amazonaws.com : ok=2    changed=0    unreachable=1    failed=0
ec2-52-36-240-171.us-west-2.compute.amazonaws.com : ok=2    changed=0    unreachable=1    failed=0
ec2-52-37-137-111.us-west-2.compute.amazonaws.com : ok=2    changed=0    unreachable=1    failed=0
ec2-52-38-193-214.us-west-2.compute.amazonaws.com : ok=2    changed=0    unreachable=1    failed=0
ec2-52-39-145-200.us-west-2.compute.amazonaws.com : ok=2    changed=0    unreachable=1    failed=0
ec2-52-39-201-67.us-west-2.compute.amazonaws.com : ok=2    changed=0    unreachable=1    failed=0
ec2-52-39-205-252.us-west-2.compute.amazonaws.com : ok=2    changed=0    unreachable=1    failed=0
ec2-52-39-88-236.us-west-2.compute.amazonaws.com : ok=2    changed=0    unreachable=1    failed=0
localhost                  : ok=33   changed=16   unreachable=0    failed=1

I updated the ansible.cfg as outline here https://community.hortonworks.com/questions/24344/aws-unreachable-error-when-executing-metron-instal... to fix similar issue but still getting the same error.

[#2]

I tried deleting the id_rsa.pub file and re-created it but still getting the error.

I created the ec2.py file where it's got the hostnames of all ec2 instances. I re-run the command with --limit @playbook.retry but not getting any results.

 ansible-playbook -i ec2.py playbook.yml --limit @playbook.retry
 [WARNING]: Host file not found: ec2.py


 [WARNING]: provided hosts list is empty, only localhost is available




PLAY [localhost] ***************************************************************
skipping: no hosts matched


PLAY [ec2] *********************************************************************
skipping: no hosts matched


PLAY [ec2] *********************************************************************
skipping: no hosts matched


PLAY [ec2] *********************************************************************
skipping: no hosts matched


PLAY [ambari_*] ****************************************************************
skipping: no hosts matched


PLAY [ambari_master] ***********************************************************
skipping: no hosts matched


PLAY [ambari_slave] ************************************************************
skipping: no hosts matched


PLAY [ambari_master] ***********************************************************
skipping: no hosts matched


PLAY [ec2] *********************************************************************
skipping: no hosts matched


PLAY [metron] ******************************************************************
skipping: no hosts matched


PLAY [hadoop_client] ***********************************************************
skipping: no hosts matched


PLAY [search] ******************************************************************
skipping: no hosts matched


ec2-52-25-211-42.us-west-2.compute.amazonaws.com
PLAY [mysql] *******************************************************************
skipping: no hosts matched


PLAY [ambari_slave] ************************************************************
skipping: no hosts matched


PLAY [sensors] *****************************************************************
skipping: no hosts matched


PLAY [enrichment] **************************************************************
skipping: no hosts matched


PLAY [web] *********************************************************************
skipping: no hosts matched


PLAY [localhost] ***************************************************************
skipping: no hosts matched


PLAY RECAP *********************************************************************

[#3]

Could you elaborate on what you meant by creating the ec2.py file? That is a file that ships with Ansible which it uses to dynamically populate the host names for the roles in the play.

[#4]

When I re-run the ansible playbook command with this option '--limit @playbook.retry', it's saying ec2.py file is not found and subsequent messages relates to ec2 hosts not found in the file. So i went ahead and created the ec2.py file with the hostnames separated by new line. It's probably a different issue just don't know what it is.

[#5]

You do not want to create an ec2.py file. That will certainly break things good. Please outline exactly what you do step-by-step when you 'start from scratch'. That should allow us to help more. Thanks!

[#6]

I'm getting this consistent error even after starting from scratch 3 times.

[#7]

@nallen Here are the steps I did the first time of deployment and subsequent retries.

1. Followed the steps here https://community.hortonworks.com/articles/26030/apache-metron-deployment-first-steps-in-the-cloud.h... to spin up EC2 instances for Metron.

2. I've got all the prerequisites nailed down with no issues.

3. After doing all the steps to deploy Metron from the above blog, it fails after running this command ansible-playbook -i ec2.py playbook.yml .

4. I wiped out the EC2 vms and re-run the command ansible-playbook -i ec2.py playbook.yml. I'm still getting the same error.

5. Tried running mvn package-DskipTests and re-run ansible-playbook -i ec2.py playbook.yml. Same issue.

[#8]

Are you sure that you exported this environment variable?

export EC2_INI_PATH=conf/ec2.ini

[#9]

Yes. I have this configured prior to running the playbook command.

[#10]

I recently added a shell script to make the process easier and to help catch common errors. If you're bold enough to try master, I'd love to see if that works for you.

If you are able to launch EC2 hosts, with your Amazon credentials, then you can start here:

https://github.com/apache/incubator-metron/tree/master/metron-deployment/amazon-ec2#deploy-metron

Thanks for giving this a go!