Failed starting metron due to zk_load_configs.sh unavailable

[#1]

Hi,

I was installing metron via ambari and it succeed, but when starting the service i got this error.

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.4.1.0/package/scripts/enrichment_master.py", line 117, in <module>
    Enrichment().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.4.1.0/package/scripts/enrichment_master.py", line 111, in restart
    self.configure(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 120, in locking_configure
    original_configure(obj, *args, **kw)
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.4.1.0/package/scripts/enrichment_master.py", line 48, in configure
    metron_service.init_zk_config(params)
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.4.1.0/package/scripts/metron_service.py", line 41, in init_zk_config
    path=ambari_format("{java_home}/bin")
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hcp/1.4.1.0-18/metron/bin/zk_load_configs.sh --zk_quorum masternode-02:2181,masternode-01:2181,insight-svr:2181 --mode PUSH --input_dir /usr/hcp/1.4.1.0-18/metron/config/zookeeper' returned 127. /bin/bash: /usr/hcp/1.4.1.0-18/metron/bin/zk_load_configs.sh: No such file or directory

Thanks.

[#2]

@Bramantya Anggriawan

Evidently the file /usr/hcp/1.4.1.0-18/metron/bin/zk_load_configs.sh seems to be missing

[#3]

@Geoffrey Shelton Okot

yes, i follow the installation guide via mabari on this link, except i installl metron on cluster which already exist. but after i install metron and tried to start it zk_load_configs.sh is not available. how do i solve this?

thanks

[#4]

@Bramantya Anggriawan

I am going through the install doc, could you tell me the size of the cluster where you are installing the Metron? Do you have dedicated nodes for Metron?
The above are just a quick view of some of the pre-requisites

[#5]

@Bramantya Anggriawan

If your installation went fine, then you should be able to execute the zookeeper from

cd $METRON_HOME/bin
./zk_load_configs.sh -i ../config/zookeeper -m PUSH -z ${zookeeper}

And remember to give the zookeeper quorum as a parameter !

[#6]

@Geoffrey Shelton Okot

here are my cluster :

• 2 masternode
• 3 workernode
• 1 database node
• 1 edge node

for metron component, i install it on workernode-01 and edge node (for UI). this because metron component has collocation rule with other component (e.g. metron parsher must on the same node with Kafka broker).

How do i check if zookeeper already have quorum since i install and start it via ambari

[#7]

@Bramantya Anggriawan

You can find out the Zookeeper details from the Ambari UI... see screenshot below.

And then you can form the zookeeper quorum for e.g. as:

host1.mydomain:2181,host2.mydomain:2181

[#8]

Btw, I wonder why you are seeing the 'No such file or directory' error if your installation went through fine. Are you able to see the file /usr/hcp/1.4.1.0-18/metron/config/zookeeper in your metron node?

Can you describe your installation procedure in detail please?

[#9]

@asubramanian

i tried to install metron via ambari with HCP-1.4.0.0-38 mpack, but failed. and i delete the service via ambari UI.

then i upgrade the mpack to HCP-1.4.1.0-18 mpack, but the repo in /etc/yum.repos.d/ambari-hdp-1.repo on each host still pointing to CP-1.4.0.0-38.

so i downlaod the repo for HCP-1.4.1.0-18 and place it on each host. after that the installation success.

there are two version folder on my /usr/hcp folder (1.4.0.0-38 and 1.4.1.0-18). and the zk_load_configs.sh are available in 1.4.0.0-38 foder but not in 1.4.0.0-38 folder.

If i want to re-install metron from the start, how to remove metron cleanly from the cluster.

thank you

[#10 · Accepted solution]

@asubramanian

As I had reiterated your installation didn't go smoothly, the workaround could be to copy all the missing folders under 1.4.0.0-38 to 1.4.0.0-18 folder and retry.

cd $METRON_HOME/bin
./zk_load_configs.sh -i ../config/zookeeper -m PUSH -z ${zookeeper}

If your setup in not on bare metal but VMware then better start from scratch .

Where is your Ambari server? If its co-related with the Master node the don't dedicate a node to only the database that would be wastage of resources make it a master

3 masternode        # the 3 zookeeper should be insatlled here 
3 workernode        # Primarily Datanodes /nodemanager/Hbase regionc servers
1 edge node

Hope that helps