Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Metron vagrant start hadoop services faiing

avatar
Master Guru

Trying to get a instance of metron up and running. Ambari is up so not sure if things are ok. I performed vagrant up and things went smoothly until task to start hadoop services. here is my error log:

TASK [ambari_config : Start All Hadoop Services node1] *************************
failed: [node1] (item=HDFS) => {"connection": "close", "content": "", "content_type": "text/plain;charset=ISO-8859-1", "failed": true, "item": "HDFS", "msg": "Status code was not [200, 202]: HTTP Error 500: Server Error", "redirected": false, "server": "Jetty(8.1.17.v20150415)", "set_cookie": "AMBARISESSIONID=tbclunfsajxy1bnyfpvk7twl;Path=/;HttpOnly", "status": 500, "url": "http://node1:8080/api/v1/clusters/metron_cluster/services/HDFS", "user": "admin"}
failed: [node1] (item=YARN) => {"connection": "close", "content": "", "content_type": "text/plain;charset=ISO-8859-1", "failed": true, "item": "YARN", "msg": "Status code was not [200, 202]: HTTP Error 500: Server Error", "redirected": false, "server": "Jetty(8.1.17.v20150415)", "set_cookie": "AMBARISESSIONID=1k3byxj6l24791q0zzxpojzmpe;Path=/;HttpOnly", "status": 500, "url": "http://node1:8080/api/v1/clusters/metron_cluster/services/YARN", "user": "admin"}
failed: [node1] (item=MAPREDUCE2) => {"connection": "close", "content": "", "content_type": "text/plain;charset=ISO-8859-1", "failed": true, "item": "MAPREDUCE2", "msg": "Status code was not [200, 202]: HTTP Error 500: Server Error", "redirected": false, "server": "Jetty(8.1.17.v20150415)", "set_cookie": "AMBARISESSIONID=onwvdexdk4x513bjb4tzj0wfz;Path=/;HttpOnly", "status": 500, "url": "http://node1:8080/api/v1/clusters/metron_cluster/services/MAPREDUCE2", "user": "admin"}
failed: [node1] (item=ZOOKEEPER) => {"connection": "close", "content": "", "content_type": "text/plain;charset=ISO-8859-1", "failed": true, "item": "ZOOKEEPER", "msg": "Status code was not [200, 202]: HTTP Error 500: Server Error", "redirected": false, "server": "Jetty(8.1.17.v20150415)", "set_cookie": "AMBARISESSIONID=1n3omygdur23a1tsoo4n49e8xt;Path=/;HttpOnly", "status": 500, "url": "http://node1:8080/api/v1/clusters/metron_cluster/services/ZOOKEEPER", "user": "admin"}
failed: [node1] (item=HBASE) => {"connection": "close", "content": "", "content_type": "text/plain;charset=ISO-8859-1", "failed": true, "item": "HBASE", "msg": "Status code was not [200, 202]: HTTP Error 500: Server Error", "redirected": false, "server": "Jetty(8.1.17.v20150415)", "set_cookie": "AMBARISESSIONID=18uowb8b50qx21vqpumqqasroq;Path=/;HttpOnly", "status": 500, "url": "http://node1:8080/api/v1/clusters/metron_cluster/services/HBASE", "user": "admin"}
failed: [node1] (item=STORM) => {"connection": "close", "content": "", "content_type": "text/plain;charset=ISO-8859-1", "failed": true, "item": "STORM", "msg": "Status code was not [200, 202]: HTTP Error 500: Server Error", "redirected": false, "server": "Jetty(8.1.17.v20150415)", "set_cookie": "AMBARISESSIONID=10s5zpez54rti1ol96n7f2orhh;Path=/;HttpOnly", "status": 500, "url": "http://node1:8080/api/v1/clusters/metron_cluster/services/STORM", "user": "admin"}
failed: [node1] (item=KAFKA) => {"connection": "close", "content": "", "content_type": "text/plain;charset=ISO-8859-1", "failed": true, "item": "KAFKA", "msg": "Status code was not [200, 202]: HTTP Error 500: Server Error", "redirected": false, "server": "Jetty(8.1.17.v20150415)", "set_cookie": "AMBARISESSIONID=g7746coyq9kindh256rofz7v;Path=/;HttpOnly", "status": 500, "url": "http://node1:8080/api/v1/clusters/metron_cluster/services/KAFKA", "user": "admin"}
to retry, use: --limit @../../playbooks/metron_full_install.retry
PLAY RECAP *********************************************************************
node1                      : ok=38changed=28   unreachable=0    failed=1

Ansible failed to complete successfully. Any error output should be

visible above. Please fix these errors and try again.

1 ACCEPTED SOLUTION

avatar
Contributor

Ok, I just saw with one with a member of the team here on Metron who has a new machine. I think the issue is our ansible scripts interacting negatively with ansible version 2.0.2. All of the members of the team here have 2.0.0.2. While this is corrected, I would suggest reverting to 2.0.0.2 and letting me know if it fixes the issue.

In case you might not know how to do that, what *I* do is remove ansible from homebrew (brew uninstall ansible) and then install directly via pip (pip install ansible==2.0.0.2).

Let me know and sorry for the bug!

View solution in original post

17 REPLIES 17

avatar
Contributor

Not an answer, but I noticed the exception in these logs is the same as https://community.hortonworks.com/questions/19734/hortonworks-2-ambari-unable-to-log-on-to-ambari-ui...

avatar
Master Guru

@lgeorge could you please provide details of how you fixed the issue so it could help others as well. I am using a macbook pro so i believe I have enough resources to bring up a vm.

avatar
Super Collaborator

@Sunile Manjee, sure. My approach is more of a workaround than a solution, but I'll do my best to describe it here.

I followed these steps while the VM was still running with Ambari and HDP deployed (the point at which the error occurred).

  1. Make a backup copy of the Vagrantfile in incubator-metron-Metron_0.1BETA_rc7/deployment/vagrant/singlenode-vagrant/
  2. Edit Vagrantfile so that ansible uses the metron_install.yml file instead of the metron_full_install.yml file:

    Near the end of the file, change the line with ansible.playbook = "../../playbooks/metron_full_install.yml"

    to

    ansible.playbook = "../../playbooks/metron_install.yml"

    Then save your changes.

  3. From the deployment/vagrant/singlenode-vagrant directory, issue the following command:

    vagrant provision

    (If the VM is stopped/halted you'd probably want to use vagrant up instead. It shouldn't hurt to run vagrant up; if it isn't needed it will let you know.)

    At this point the Metron installation playbook stepped through nine blocks of instructions; looks they're called "plays". You can see the list in the metron_install.yml file, and you can look in the roles directory for a better understanding of the tasks associated with the role in each play (hadoop_setup, mysql-server, etc.).

When the provisioning finished I could see all four topologies in the Storm UI but I didn't see any data. I'd gotten an error during the hadoop_setup role (under hosts: hadoop_client) so I re-ran that part by making a backup copy of metron_install.yml, removing everything except the hadoop_client section from the original file, and running vagrant provision again. A little while later I started seeing data in the Metron UI -- great to see it!

I hope I've remembered everything; let me know if any of it isn't clear.

avatar
New Contributor

I went through the same approach and still running into a few more failures. 1) HDFS 2) HBase table creation and 3) install exited after "Build snort" task. WIll try again this mrng and update.

avatar
Contributor

Ok, I just saw with one with a member of the team here on Metron who has a new machine. I think the issue is our ansible scripts interacting negatively with ansible version 2.0.2. All of the members of the team here have 2.0.0.2. While this is corrected, I would suggest reverting to 2.0.0.2 and letting me know if it fixes the issue.

In case you might not know how to do that, what *I* do is remove ansible from homebrew (brew uninstall ansible) and then install directly via pip (pip install ansible==2.0.0.2).

Let me know and sorry for the bug!

avatar
Super Collaborator

on my new mac book pro, i saw the same issue this morning. I followed casey's directions and it fixed it for me. Until we have a better fix, I have updated the documentation to instruction ansible 2.0.0.2 be installed): https://community.hortonworks.com/articles/24818/metron-tech-preview-1-install-instructions-on-sing....

avatar
Guru

Yes. This is the issue. Once i switched to 2.0.0.2 of ansible, I was able to go all the way. Earlier version was 2.0.2.0 and not 2.2 though. Thanks!

avatar
Contributor

Yes! 2.0.2, not 2.2 I misstyped. I will correct the original answer.