Issue when installing apache metron on sandbox (Ambari)

[#1]

Hey everyone,

I'm trying to install apache metron on Hortonworks HDP sandbox via Ambari.

I have issue on step " Metron Enrichment Start " and ha

ve this view :

And i have this Traceback :

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/enrichment_master.py", line 113, in <module>
    Enrichment().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/enrichment_master.py", line 64, in start
    commands.init_kafka_topics()
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/enrichment_commands.py", line 127, in init_kafka_topics
    metron_service.init_kafka_topics(self.__params, [self.__enrichment_topic, self.__params.metron_error_topic])
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/metron_service.py", line 120, in init_kafka_topics
    user=params.kafka_user)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hdp/current/kafka-broker/bin/kafka-topics.sh                             --zookeeper sandbox.hortonworks.com:2181                             --create                             --if-not-exists                             --topic enrichments                             --partitions 1                             --replication-factor 1                             --config retention.bytes=10737418240' returned 1. OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Error while executing topic command : replication factor: 1 larger than available brokers: 0
[2017-05-22 12:11:31,848] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: replication factor: 1 larger than available brokers: 0
 (kafka.admin.TopicCommand$)

If someone could help me please ? It's the second time i have this issue.

PS: I run sandbox 2.6 in a docker

[#2]

Not 100% sure, but seems like a resource constraints issue. Have you tried running this again, getting the same result? Also, what kind of resources (CPU, RAM, etc.) are you allocating to this?

[#3]

Yes, if run again i get same results... I allocate 1024Mo Ram

[#4 · Accepted solution]

@Jon Zeolla - is on the right track. Your Kafka broker seems to have died and as a result, the Metron service start process cannot contact it to create the topics it needs. I'd recommend either using Metron's Full-Dev environment. If you're determined to get it running on the Sandbox, you'll need to do a resource sizing activity to make sure all the services can live in the amount of memory available to the container.

[#5]

Thanks you ! I will try the Metron's Full-Dev.

[#6]

@David Lyle , i only have a Cloud machine. Of course, when in run full-dev metron in my VM i get

"Stderr: VBoxManage: error: VT-x is not available (VERR_VMX_NO_VMX)" because nested virtualisation..

Do you have any idea to fix it?

[#7]

I only took a quick look into this - it seems that as of Jun 3, 2014 AWS doesn't seem to support VT-x.

[#8]

solved by using docker method