question Re: Fastcapa Test Environment not work in Metron 0.3.0 in Archives of Support Questions (Read Only)

Fastcapa Test Environment not work in Metron 0.3.0

naveedanjum — Fri, 30 Dec 2016 13:41:06 GMT

Hi,

On running up the vagrant up I am getting this error

sink: Running ansible-playbook...ERROR! role definitions must contain a role name

The error appears to have been in '/root/incubator-metron/metron-deployment/vagrant/fastcapa-test-platform/playbook.yml': line 30, column 7, but maybe elsewhere in the file depending on the exact syntax problem.The offending line appears to be:  - { role: sensor-test-mode, pcap_replay: True, install_yaf: False, install_snort: False }  - service: name=pcap-replay state=started  ^ hereAnsible failed to complete successfully. Any error output should bevisible above. Please fix these errors and try again.

Re: Fastcapa Test Environment not work in Metron 0.3.0

sball — Fri, 30 Dec 2016 19:33:34 GMT

Could you please confirm your ansible version? This is likely a version conflict with Ansible. We recommend 2.0.0.2

Re: Fastcapa Test Environment not work in Metron 0.3.0

naveedanjum — Mon, 02 Jan 2017 13:43:35 GMT

I can confirm that following are my plateform information

ansible 2.0.0.2
  config file = 
  configured module search path = Default w/o overrides
--
Vagrant 1.8.1
--
Python 2.7.11
--
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T21:41:47+05:00)
Maven home: /usr/local/maven
Java version: 1.8.0_101, vendor: Oracle Corporation
Java home: /opt/jdk1.8.0_101/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-642.4.2.el6.x86_64", arch: "amd64", family: "unix"
--
Linux centos16 2.6.32-642.4.2.el6.x86_64 #1 SMP Tue Aug 23 19:58:13 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Ansible is 2.0.0.2 ? Any Idea what could be the potential problem ?

Re: Fastcapa Test Environment not work in Metron 0.3.0

naveedanjum — Mon, 02 Jan 2017 13:53:09 GMT

Following is the output I am getting after enabling ansible verbosity mode for error detection

sink: Running ansible-playbook...
PYTHONUNBUFFERED=1 ANSIBLE_NOCOLOR=true ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ControlMaster=auto -o ControlPersist=60s' ansible-playbook --connection=ssh --timeout=30 --limit='all' --inventory-file=/root/incubator-metron/metron-deployment/vagrant/fastcapa-test-platform/.vagrant/provisioners/ansible/inventory -vvv playbook.yml
Using /root/incubator-metron/metron-deployment/vagrant/fastcapa-test-platform/ansible.cfg as config file
ERROR! role definitions must contain a role name

The error appears to have been in '/root/incubator-metron/metron-deployment/vagrant/fastcapa-test-platform/playbook.yml': line 30, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

    - { role: sensor-test-mode, pcap_replay: True, install_yaf: False, install_snort: False }
    - service: name=pcap-replay state=started
      ^ here
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

@Simon Elliston Ball

Re: Fastcapa Test Environment not work in Metron 0.3.0

nallen — Wed, 04 Jan 2017 01:52:30 GMT

Farrukh, you've found a bug for which I have created METRON-645. Thanks so much for highlighting the problem.

I have submitted a fix as PR 401. Feel free to give it a go.

Re: Fastcapa Test Environment not work in Metron 0.3.0

naveedanjum — Wed, 04 Jan 2017 17:22:06 GMT

Thanks @nallen, Let me try

Re: Fastcapa Test Environment not work in Metron 0.3.0

naveedanjum — Thu, 05 Jan 2017 15:02:59 GMT

Thank @nallen you fix work correctly I am able to run fastcap test now.

Following is the output

TASK [debug] *******************************************************************
ok: [source] => {
    "msg": "Successfully received a Kafka message from fastcapa!"
}

PLAY RECAP *********************************************************************
sink                       : ok=41   changed=25   unreachable=0    failed=0   
source                     : ok=46   changed=26   unreachable=0    failed=0

Only problem that i am facing right now is that i am unable to access Ambari on port 8080 or Kibana on 5000 (For Fastcapa Test Environment) where as in quick development and codelab it is working. Can you please tell me what could be the cause ?

Furthermore, I am unable to execute the following command on sink

/usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic pcap

It says it does not exists

-bash: /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh: No such file or directory

Re: Fastcapa Test Environment not work in Metron 0.3.0

nallen — Thu, 05 Jan 2017 22:22:20 GMT

The `fastcapa-test-environment` is only designed to test Fastcapa. It does not install Ambari or Kibana.

The Kafka Broker is installed on the `source` host. You need to `vagrant ssh source`.

You can see what is installed and where by looking at `metron-deployment/vagrant/fastcapa-test-platform/playbook.yml`.

Re: Fastcapa Test Environment not work in Metron 0.3.0

naveedanjum — Tue, 10 Jan 2017 13:26:27 GMT

Hi, Thanks @nallen

Can you please guide ? How can I install Kibana and Ambari along with it ? What changes needs to applied ? I would suggest we may need to add these service on Source Machine. As, without Ambari and Kibana, we are unable to visualize and represent information. Any work around for this ?

Re: Fastcapa Test Environment not work in Metron 0.3.0

nallen — Tue, 10 Jan 2017 22:22:42 GMT

The `fastcapa-test-environment` was made specifically for testing Fastcapa only. I would suggest that you install Metron using one of the other installation methods and then after you have Metron installed, install Fastcapa. That will be much easier than installing Metron on top of `fastcapa-test-environment`.

Re: Fastcapa Test Environment not work in Metron 0.3.0

naveedanjum — Wed, 11 Jan 2017 13:30:04 GMT

Hi, Thanks @nallen

How would I redirect pcap data from fastcapa-test-enviorment to say quick-development VM's ?

Re: Fastcapa Test Environment not work in Metron 0.3.0

nallen — Wed, 11 Jan 2017 23:30:18 GMT

I doubt that is possible. At least, I don't know how to do it. Part of the Vagrant definition for each of these environments defines the virtual network that is used by each VM. I am not sure how you would bridge the two networks after each of these environments are already running.

You could definitely create you own Vagrant definition that accomplishes this, but it is not something that can be done with the existing environments.

Re: Fastcapa Test Environment not work in Metron 0.3.0

gauravb117 — Mon, 19 Aug 2019 12:01:30 GMT

Cant find any solution to this !!