Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

2 ambari clusters in one big cluster

SOLVED Go to solution
Highlighted

2 ambari clusters in one big cluster

hi all


I want to hear from hortonworks guys about the following idea


we want to build the following architecture


1. ambari cluster that include ( HDP 3.1 )

3-5 kafka machines

3 zookeper servers that are dedicated only for the kafka machines


2. ambari cluster that include all other application ( HDP 3.1 )

HDFS - service

YARN - service

MAPREDUCE - service

HIVE - service

SPARK

3 Zookeepers that dedicated to hdfs , yarn , etc

data nodes machines


so both cluster should talk with each other like one big cluster


is this possible ?


Michael-Bronson
1 ACCEPTED SOLUTION

Accepted Solutions

Re: 2 ambari clusters in one big cluster

Super Mentor

@Michael Bronson

Regarding your query: "2 ambari clusters in one big cluster"

Ambari can currently manage only one cluster deployed to it. So from ambari perspective you manage two cluster from same Ambari.

However, You can run two clusters running separately and still be able to establish communication between Kafka cluster to the other cluster which has all the services running.

Like Spark running in Cluster two can interact with External Kafka Broker running inside Cluster one.

4 REPLIES 4

Re: 2 ambari clusters in one big cluster

Super Mentor

@Michael Bronson

Regarding your query: "2 ambari clusters in one big cluster"

Ambari can currently manage only one cluster deployed to it. So from ambari perspective you manage two cluster from same Ambari.

However, You can run two clusters running separately and still be able to establish communication between Kafka cluster to the other cluster which has all the services running.

Like Spark running in Cluster two can interact with External Kafka Broker running inside Cluster one.

Re: 2 ambari clusters in one big cluster

@DEAR Jay "clusters running separately and still be able to establish communication between Kafka cluster to the other cluster which has all the services "

can you please explain in basic details what are the configuration on both cluster to enable contamination between one ambari cluster to the ambari cluster that include the kafka?

Michael-Bronson

Re: 2 ambari clusters in one big cluster

Super Mentor

@Michael Bronson

What i mean to say is this ::: At AmbariServer level you can not manage two clusters as a Single cluster.

But if you want to Produce Messages to Kafka (For example via Spark Application then that can be done even in Non Ambari Managed cluster as well right)

For Example: In your Spark application define the correct Kafka Broker host/port ...etc like following:

https://spark.apache.org/docs/latest/streaming-kafka-0-10-integration.html



Re: 2 ambari clusters in one big cluster

Community Manager

The above question and the entire reply thread below was originally posted in the Community Help track. On Tue Jul 2 01:01 UTC 2019, a member of the HCC moderation staff moved it to the Design & Architecture track. The Community Help Track is intended for questions about using the HCC site itself, not technical questions.