Support Questions

Find answers, ask questions, and share your expertise

Metron "metron_elasticsearch_templates : Wait for Index to Become Available" failure on physical deployment

avatar
Explorer

I'm running Metron on an 8 node physical cluster (CentOS 6), with a virtual installer node (CentOS 7).

I'm running into the following error after running `ansible-playbook -i /root/incubator-metron/metron-deployment/inventory/$INSTALL_CONF_DIRECTORY metron_install.yml --skip-tags="solr,sensors"`:

TASK [metron_elasticsearch_templates : Wait for Index to Become Available] *****

fatal: [server2]: FAILED! => {"failed": true, "msg": "ERROR! The conditional check 'result.content.find(\"green\") != -1 or result.content.find(\"yellow\") != -1' failed. The error was: ERROR! error while evaluating conditional (result.content.find(\"green\") != -1 or result.content.find(\"yellow\") != -1): ERROR! 'dict object' has no attribute 'content'"}

fatal: [server2]: FAILED! => {"failed": true, "msg": "ERROR! The conditional check 'result.content.find(\"green\") != -1 or result.content.find(\"yellow\") != -1' failed. The error was: ERROR! error while evaluating conditional (result.content.find(\"green\") != -1 or result.content.find(\"yellow\") != -1): ERROR! 'dict object' has no attribute 'content'"}

However when I check the servers manually I see:

[root@e104d0xxxxxx scripts]# curl server1:9200/_cluster/health

{"cluster_name":"metron","status":"green","timed_out":false,"number_of_nodes":2,"number_of_data_nodes":2,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}[root@e104d0553817 scripts]#

[root@e104d0xxxxxx scripts]# curl server2:9200/_cluster/health

{"cluster_name":"metron","status":"green","timed_out":false,"number_of_nodes":2,"number_of_data_nodes":2,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}[root@e104d0553817 scripts]#

And when I check in my web browser, I get:

1473215295 22:28:15 metron green 2 2 0 0 0 0 0 0 - 100.0%

1473215299 22:28:19 metron green 2 2 0 0 0 0 0 0 - 100.0%

I've tried editing incubator-metron/metron-deployment/roles/metron_elasticsearch_templates/tasks/load_templates.yml and increase the retries (up to 600) and delays (up to 1000) with no success.

Some details:

Metron 0.2.0BETA

--

* (no branch)

--

commit 75642001803396e8884385b0fc297a2312ead3eb

Author: cstella <cestella@gmail.com>

Date: Wed Jul 13 11:15:20 2016 -0400

METRON-298 Remove the effective_tld_names.dat files. closes apache/incubator-metron#186

--

metron-deployment/playbooks/metron_install.yml | 45 ++++++++++++++++++++

.../roles/ambari_common/tasks/main.yml | 5 ++-

.../roles/elasticsearch/defaults/main.yml | 2 +-

.../roles/elasticsearch/tasks/elasticsearch.yml | 2 +-

metron-deployment/roles/httplib2/tasks/main.yml | 3 +-

.../roles/kibana/tasks/elasticdump.yml | 4 +-

.../roles/opentaxii/tasks/opentaxii.yml | 2 +-

7 files changed, 57 insertions(+), 6 deletions(-)

--

ansible 2.0.0.2

config file = /root/ansible.cfg

configured module search path = ../extra_modules

--

./platform-info.sh: line 52: vagrant: command not found

--

Python 2.6.6

--

Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 2014-12-14T17:29:23+00:00)

Maven home: /opt/maven

Java version: 1.8.0_91, vendor: Oracle Corporation

Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.91-1.b14.el6.x86_64/jre

Default locale: en_US, platform encoding: UTF-8

OS name: "linux", version: "3.10.0-327.el7.x86_64", arch: "amd64", family: "unix"

--

Linux e104d0553817 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

1 ACCEPTED SOLUTION

avatar
Explorer

The issue was because I had set environment variables to use our proxies and it was trying to proxy the cluster health check, but when I did it at the command line that env var wasn't set. I set no_proxy for server1,server2 and it worked properly.

View solution in original post

1 REPLY 1

avatar
Explorer

The issue was because I had set environment variables to use our proxies and it was trying to proxy the cluster health check, but when I did it at the command line that env var wasn't set. I set no_proxy for server1,server2 and it worked properly.