Apache Metron Single Node Installation Issue: Ambari Client Exception

[#1]

Hi,

I'm trying to install Metron on a single node, following the instructions here:

https://github.com/apache/incubator-metron/tree/master/metron-deployment/vagrant/full-dev-platform

I receive the following error, indicating an issue with Ambari:

fatal: [node1]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_args": {"blueprint_name": null, "blueprint_var": null, "cluster_name": "metron_cluster", "cluster_state": "started", "configurations": null, "host": "node1", "password": "admin", "port": 8080, "username": "admin", "wait_for_complete": true}, "module_name": "ambari_cluster_state"}, "msg": "Ambari client exception occurred: No JSON object could be decoded"}

Note that Ambari is up and running. I can log into it using a browser.

Here's the output of platform-info.sh:

Metron 0.2.0BETA -- * master -- commit c85c74269327487a5fe607ea85ebc56d3b5650ef Author: cstella <cestella@gmail.com> Date: Mon Sep 26 15:14:53 2016 -0400 METRON-452: Add rudimentary configuration management functions to Stellar closes apache/incubator-metron#269 -- metron-deployment/vagrant/full-dev-platform/Vagrantfile | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) -- ansible 2.0.0.2 config file = <MY_FOLDER>/incubator-metron/metron-deployment/vagrant/full-dev-platform/ansible.cfg configured module search path = ../../extra_modules -- Vagrant 1.8.5 -- Python 2.7.10 -- Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T08:41:47-08:00) Maven home: /usr/local/Cellar/maven/3.3.9/libexec Java version: 1.8.0_73, vendor: Oracle Corporation Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_73.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x", version: "10.10.5", arch: "x86_64", family: "mac" -- Darwin <MY_MAC>.local 14.5.0 Darwin Kernel Version 14.5.0: Mon Aug 29 21:14:16 PDT 2016; root:xnu-2782.50.6~1/RELEASE_X86_64 x86_64

It is perhaps unrelated but worth mentioning that I have made the following changes in my Vagrantfile:

1) set the number of cpus to 2 instead of the default 4.

2) set config.ssh.insert_key = false

I appreciate your help.

[#2]

I should add that I receive the mentioned error during the following task:

TASK [ambari_config : Start the ambari cluster - wait] *************************

[#3]

Hey HS,

I think this is most likely an issue with master. I am seeing the same error on my box as well.

<node1> ESTABLISH SSH CONNECTION FOR USER: vagrant
<node1> SSH: EXEC ssh -C -q -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -i /Users/asubramanian/Desktop/Metron/incubator-metron-fork/incubator-metron/metron-deployment/vagrant/full-dev-platform/.vagrant/machines/node1/virtualbox/private_key -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=30 -o ControlPath=/Users/asubramanian/.ansible/cp/%h-%p-%r -tt node1 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-gavsahqeydndosbmwnmihimsyouqkvhh; LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/vagrant/.ansible/tmp/ansible-tmp-1474969027.04-192863002921867/ambari_cluster_state; rm -rf "/home/vagrant/.ansible/tmp/ansible-tmp-1474969027.04-192863002921867/" > /dev/null 2>&1'"'"'"'"'"'"'"'"''"'"''
fatal: [node1]: FAILED! => {"changed": false, "failed": true, "invocation": {"module_args": {"blueprint_name": null, "blueprint_var": null, "cluster_name": "metron_cluster", "cluster_state": "started", "configurations": null, "host": "node1", "password": "admin", "port": 8080, "username": "admin", "wait_for_complete": true}, "module_name": "ambari_cluster_state"}, "msg": "Ambari client exception occurred: No JSON object could be decoded"}

Here's my platform info output:

(testmetron) asubramanian:incubator-metron asubramanian$ metron-deployment/scripts/platform-info.sh
Metron 0.2.0BETA
--
* master
--
commit 3d5f279caea2b9349a05b3a7316ef19e2ca8cb11
Author: cstella <cestella@gmail.com>
Date:   Mon Sep 26 18:00:22 2016 -0400
    METRON-453: Add a stellar shell function to open an external editor and return the editor's contents closes apache/incubator-metron#272
--
 metron-deployment/inventory/metron_example/hosts         | 6 +++---
 metron-deployment/vagrant/full-dev-platform/Vagrantfile  | 1 +
 metron-deployment/vagrant/quick-dev-platform/Vagrantfile | 1 +
 metron-deployment/vagrant/quick-dev-platform/run.sh      | 2 +-
 4 files changed, 6 insertions(+), 4 deletions(-)
--
ansible 2.0.0.2
  config file =
  configured module search path = Default w/o overrides
--
Vagrant 1.8.1
--
Python 2.7.10
--
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T22:11:47+05:30)
Maven home: /usr/local/Cellar/maven/3.3.9/libexec
Java version: 1.8.0_91, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.11.6", arch: "x86_64", family: "mac"
--
Darwin asubramanian.local 15.6.0 Darwin Kernel Version 15.6.0: Mon Aug 29 20:21:34 PDT 2016; root:xnu-3248.60.11~1/RELEASE_X86_64 x86_64

IMHO this calls for logging a defect.

Note that you can make the following changes to get a verbose output when you run provision.

- Edit the Vagrant file

- Add "ansible.verbose = "vvv"" in the bottom of the file where the other parameters are defined. Something like this:

<snip>
    ansible.inventory_path = "../../inventory/full-dev-platform"
    ansible.verbose = "vvv"
  end
end

Regards,

Anand

[#4 · Accepted solution]

Hi,

It seems that the Ambari client gets unhappy when you ask it to start services and they are all started. I have opened METRON-466 to track it.

In the meantime, you should be able to comment out lines 34-42 in metron-deployment/roles/ambari_config/start_hdp.yml to get you going. I hope to have a fix up shortly.

Thanks!

-D...

[#5]

FWIW I was able to run quick-dev installation without any issues.

[#6]

Thanks for your response. The issue was resolved and Metron is successfully deployed.

[#7]

Glad to hear it. If you haven't, would you mind accepting the answer that helped?

Thanks!

-D...

[#8]

I am trying to install metron on ubutnu 16.04 server and receiving following error. any help would be appreciated\. THanks

TASK [ambari_config : Deploy cluster with Ambari; http://node1:8080] *********** fatal: [node1]: FAILED! => {"changed": false, "failed": true, "msg": "value of wait_for_complete must be one of: y,yes,on,1,true,1,True,n,no,off,0,false,0,False, got: True"} to retry, use: --limit @/home/rendev/Downloads/incubator-metron-Metron_0.2.1BETA_rc2/metron-deployment/playbooks/metron_full_install.retry PLAY RECAP ********************************************************************* node1 : ok=31 changed=23 unreachable=0 failed=1 Ansible failed to complete successfully. Any error output should be visible above. Please fix these errors and try again.

,

I am trying to install metron on ubutnu 16.04 server and receiving following error. any help would be appreciated:

TASK [ambari_config : Deploy cluster with Ambari; http://node1:8080] *********** fatal: [node1]: FAILED! => {"changed": false, "failed": true, "msg": "value of wait_for_complete must be one of: y,yes,on,1,true,1,True,n,no,off,0,false,0,False, got: True"} to retry, use: --limit @/home/rendev/Downloads/incubator-metron-Metron_0.2.1BETA_rc2/metron-deployment/playbooks/metron_full_install.retry PLAY RECAP ********************************************************************* node1 : ok=31 changed=23 unreachable=0 failed=1 Ansible failed to complete successfully. Any error output should be visible above. Please fix these errors and try again.