Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

METRON FAILING TO START AFTER RESETING AMBARI SERVER AND CHANGING THE CLUSTER NAME

METRON FAILING TO START AFTER RESETING AMBARI SERVER AND CHANGING THE CLUSTER NAME

New Contributor

esource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hcp/1.4.1.0-18/metron/bin/zk_load_configs.sh --zk_quorum node02.local:2181,node03.local:2181,node01.local:2181 --mode PATCH --config_type GLOBAL --patch_file /tmp/metron-global-config-patch.json' returned 1. 2018-03-07 13:00:41 ERROR ConfigurationManager:314 - Unable to apply patch to Zookeeper config

org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /metron/topology/global
8 REPLIES 8

Re: METRON FAILING TO START AFTER RESETING AMBARI SERVER AND CHANGING THE CLUSTER NAME

Contributor

Hi @abu ameen,

Are you able to access to Zookeeper CLI and run a ls?

What do you mean about "reseting Ambari Server"?

Gonçalo

Re: METRON FAILING TO START AFTER RESETING AMBARI SERVER AND CHANGING THE CLUSTER NAME

New Contributor

Hi @Gonçalo Cunha

i reseted ambari server using ambari-server reset everything was gone then i deployed the cluster from scratch using ambari and encounterd the above issue

ZK cli:

[zk: localhost:2181(CONNECTED) 6] ls

[zk: localhost:2181(CONNECTED) 7]

Re: METRON FAILING TO START AFTER RESETING AMBARI SERVER AND CHANGING THE CLUSTER NAME

Mentor

@abu ameen,

The command is wrong note the slash

 [zk: localhost:2181(CONNECTED) 6] ls /


Re: METRON FAILING TO START AFTER RESETING AMBARI SERVER AND CHANGING THE CLUSTER NAME

Contributor
@abu ameen

I think there's the problem. Zookeeper doesn't have any paths for Metron like -> /metron/topology/global.

You can create this specific path but you can face further issues such as ACLs, etc. or you can reinstall Metron.

Gonçalo

Highlighted

Re: METRON FAILING TO START AFTER RESETING AMBARI SERVER AND CHANGING THE CLUSTER NAME

New Contributor

@abu ameen

Did you get this to work?
I'm expriencing the same issue - with HDP 2.6 and HCP 1.4.2.

Metron Enrichment fails to start.

Showing this same error.

Reinstalled Metron - still the same.

,

@abu ameen

Did you ever get this to work?
I tried reinstalling metron, no luck.

Re: METRON FAILING TO START AFTER RESETING AMBARI SERVER AND CHANGING THE CLUSTER NAME

New Contributor

@Mark Kotlyar

yes i did.

could you try running the Zookeeper CLI (usr/hdp/current/zookeeper-client/zkCli.sh ) and run a (ls /) or (ls /metron/topolgoy) ?

what do you get ?

Re: METRON FAILING TO START AFTER RESETING AMBARI SERVER AND CHANGING THE CLUSTER NAME

New Contributor

Yeah. Did that. Nothing was there.

I tried this:

  1. ./zk_load_configs.sh -i ../config/zookeeper -m PUSH -z ${zookeeper}

So far, it seems I got past the error you had here (and me) in the Enrichment.

Now waiting for the rest of the services to startup.

Fingers crossed :D

Re: METRON FAILING TO START AFTER RESETING AMBARI SERVER AND CHANGING THE CLUSTER NAME

Contributor

@abu ameen

Most probably there's too many open connections to Zookeeper on that host. Zookeeper limits the maximum possible connections from a single host. Per default this is 60. If there are 60 open connections to zookeeper from a single host, no further connections are opened. This is why the script fails.

To check from where the connections are opened execute:

netstat -nape | awk '{if ($5 == "ADD_MY_ZOOKEEPER_IP_HERE:2181") print $4, $9;}' | sort | uniq -c

Identify the process ids of the applications opening connections:

1 192.168.123.1:45162 577/java

In this case it's 577

Check which application this is. It might be the Metron REST server. In this case you can kill it and restart it.