Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How can multiple instance of apache-slider application be run on yarn

How can multiple instance of apache-slider application be run on yarn

New Contributor

I have a kafka java hbase combine application, and using slider I am running it in yarn.

How can I run multiple instance of this application.

4 REPLIES 4

Re: How can multiple instance of apache-slider application be run on yarn

Rising Star

You should be able to create a new application with a different name using the same templates. The main things to watch out for are port conflicts and ensuring different applications are using different HDFS and ZK paths. These should already be taken care of if you copied the hbase and kafka templates from the slider app-packages.

Re: How can multiple instance of apache-slider application be run on yarn

Community Manager

The above question and the entire response thread below was originally misposted in the Community Help track. On Thu Jul 4 21:59 UTC 2019, a member of the HCC moderation staff moved it to the Data Ingestion & Streaming track. The Community Help Track is intended for questions about using the HCC site itself, not technical questions about the Kafka, Hbase, Slider or YARN.

Re: How can multiple instance of apache-slider application be run on yarn

New Contributor

Thank you @billie for your answer.

As I have to create multiple instances of same one application, HDFS and ZK paths cannot be different.

And also, add in problem description that -

Our application is consuming data from kafka topic which have multiple partitions, and our application have only one instance so have only one consumer to consume a large amount of data. That's why need multiple instance of our application.


Re: How can multiple instance of apache-slider application be run on yarn

Rising Star

Slider provides variables here and here that will give you unique HDFS and ZK paths for each application, so if you're using those it will be fine.