Fastcapa Test Environment not work in Metron 0.3.0

[#1]

Hi,

On running up the vagrant up I am getting this error

sink: Running ansible-playbook...ERROR! role definitions must contain a role name

The error appears to have been in '/root/incubator-metron/metron-deployment/vagrant/fastcapa-test-platform/playbook.yml': line 30, column 7, but maybe elsewhere in the file depending on the exact syntax problem.The offending line appears to be:  - { role: sensor-test-mode, pcap_replay: True, install_yaf: False, install_snort: False }  - service: name=pcap-replay state=started  ^ hereAnsible failed to complete successfully. Any error output should bevisible above. Please fix these errors and try again.

[#2]

Could you please confirm your ansible version? This is likely a version conflict with Ansible. We recommend 2.0.0.2

[#3]

I can confirm that following are my plateform information

ansible 2.0.0.2
  config file = 
  configured module search path = Default w/o overrides
--
Vagrant 1.8.1
--
Python 2.7.11
--
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T21:41:47+05:00)
Maven home: /usr/local/maven
Java version: 1.8.0_101, vendor: Oracle Corporation
Java home: /opt/jdk1.8.0_101/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-642.4.2.el6.x86_64", arch: "amd64", family: "unix"
--
Linux centos16 2.6.32-642.4.2.el6.x86_64 #1 SMP Tue Aug 23 19:58:13 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Ansible is 2.0.0.2 ? Any Idea what could be the potential problem ?

[#4]

Following is the output I am getting after enabling ansible verbosity mode for error detection

sink: Running ansible-playbook...
PYTHONUNBUFFERED=1 ANSIBLE_NOCOLOR=true ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ControlMaster=auto -o ControlPersist=60s' ansible-playbook --connection=ssh --timeout=30 --limit='all' --inventory-file=/root/incubator-metron/metron-deployment/vagrant/fastcapa-test-platform/.vagrant/provisioners/ansible/inventory -vvv playbook.yml
Using /root/incubator-metron/metron-deployment/vagrant/fastcapa-test-platform/ansible.cfg as config file
ERROR! role definitions must contain a role name

The error appears to have been in '/root/incubator-metron/metron-deployment/vagrant/fastcapa-test-platform/playbook.yml': line 30, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

    - { role: sensor-test-mode, pcap_replay: True, install_yaf: False, install_snort: False }
    - service: name=pcap-replay state=started
      ^ here
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

@Simon Elliston Ball

[#5 · Accepted solution]

Farrukh, you've found a bug for which I have created METRON-645. Thanks so much for highlighting the problem.

I have submitted a fix as PR 401. Feel free to give it a go.

[#6]

Thanks @nallen, Let me try

[#7]

Thank @nallen you fix work correctly I am able to run fastcap test now.

Following is the output

TASK [debug] *******************************************************************
ok: [source] => {
    "msg": "Successfully received a Kafka message from fastcapa!"
}

PLAY RECAP *********************************************************************
sink                       : ok=41   changed=25   unreachable=0    failed=0   
source                     : ok=46   changed=26   unreachable=0    failed=0   

Only problem that i am facing right now is that i am unable to access Ambari on port 8080 or Kibana on 5000 (For Fastcapa Test Environment) where as in quick development and codelab it is working. Can you please tell me what could be the cause ?

Furthermore, I am unable to execute the following command on sink

/usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic pcap

It says it does not exists

-bash: /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh: No such file or directory

[#8]

The `fastcapa-test-environment` is only designed to test Fastcapa. It does not install Ambari or Kibana.

The Kafka Broker is installed on the `source` host. You need to `vagrant ssh source`.

You can see what is installed and where by looking at `metron-deployment/vagrant/fastcapa-test-platform/playbook.yml`.

[#9]

Hi, Thanks @nallen

Can you please guide ? How can I install Kibana and Ambari along with it ? What changes needs to applied ? I would suggest we may need to add these service on Source Machine. As, without Ambari and Kibana, we are unable to visualize and represent information. Any work around for this ?

[#10]

The `fastcapa-test-environment` was made specifically for testing Fastcapa only. I would suggest that you install Metron using one of the other installation methods and then after you have Metron installed, install Fastcapa. That will be much easier than installing Metron on top of `fastcapa-test-environment`.