Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Issue when installing apache metron on sandbox (Ambari)

avatar

Hey everyone,

I'm trying to install apache metron on Hortonworks HDP sandbox via Ambari.

I have issue on step " Metron Enrichment Start " and ha

ve this view :

15637-capture.png

And i have this Traceback :

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/enrichment_master.py", line 113, in <module>
    Enrichment().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/enrichment_master.py", line 64, in start
    commands.init_kafka_topics()
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/enrichment_commands.py", line 127, in init_kafka_topics
    metron_service.init_kafka_topics(self.__params, [self.__enrichment_topic, self.__params.metron_error_topic])
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/metron_service.py", line 120, in init_kafka_topics
    user=params.kafka_user)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hdp/current/kafka-broker/bin/kafka-topics.sh                             --zookeeper sandbox.hortonworks.com:2181                             --create                             --if-not-exists                             --topic enrichments                             --partitions 1                             --replication-factor 1                             --config retention.bytes=10737418240' returned 1. OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Error while executing topic command : replication factor: 1 larger than available brokers: 0
[2017-05-22 12:11:31,848] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: replication factor: 1 larger than available brokers: 0
 (kafka.admin.TopicCommand$)

If someone could help me please ? It's the second time i have this issue.

PS: I run sandbox 2.6 in a docker

1 ACCEPTED SOLUTION

avatar
Expert Contributor

@Jon Zeolla - is on the right track. Your Kafka broker seems to have died and as a result, the Metron service start process cannot contact it to create the topics it needs. I'd recommend either using Metron's Full-Dev environment. If you're determined to get it running on the Sandbox, you'll need to do a resource sizing activity to make sure all the services can live in the amount of memory available to the container.

View solution in original post

7 REPLIES 7

avatar
Explorer

Not 100% sure, but seems like a resource constraints issue. Have you tried running this again, getting the same result? Also, what kind of resources (CPU, RAM, etc.) are you allocating to this?

avatar

Yes, if run again i get same results... I allocate 1024Mo Ram

avatar
Expert Contributor

@Jon Zeolla - is on the right track. Your Kafka broker seems to have died and as a result, the Metron service start process cannot contact it to create the topics it needs. I'd recommend either using Metron's Full-Dev environment. If you're determined to get it running on the Sandbox, you'll need to do a resource sizing activity to make sure all the services can live in the amount of memory available to the container.

avatar

Thanks you ! I will try the Metron's Full-Dev.

avatar

@David Lyle , i only have a Cloud machine. Of course, when in run full-dev metron in my VM i get

"Stderr: VBoxManage: error: VT-x is not available (VERR_VMX_NO_VMX)" because nested virtualisation..

Do you have any idea to fix it?

avatar
Explorer

I only took a quick look into this - it seems that as of Jun 3, 2014 AWS doesn't seem to support VT-x.

avatar

solved by using docker method