Created on 07-03-2019 02:04 PM - edited 09-16-2022 07:29 AM
I have a kafka java hbase combine application, and using slider I am running it in yarn.
How can I run multiple instance of this application.
Created 07-03-2019 03:25 PM
You should be able to create a new application with a different name using the same templates. The main things to watch out for are port conflicts and ensuring different applications are using different HDFS and ZK paths. These should already be taken care of if you copied the hbase and kafka templates from the slider app-packages.
Created 07-04-2019 10:00 PM
The above question and the entire response thread below was originally misposted in the Community Help track. On Thu Jul 4 21:59 UTC 2019, a member of the HCC moderation staff moved it to the Data Ingestion & Streaming track. The Community Help Track is intended for questions about using the HCC site itself, not technical questions about the Kafka, Hbase, Slider or YARN.
Created 07-11-2019 12:52 PM
Thank you @billie for your answer.
As I have to create multiple instances of same one application, HDFS and ZK paths cannot be different.
And also, add in problem description that -
Our application is consuming data from kafka topic which have multiple partitions, and our application have only one instance so have only one consumer to consume a large amount of data. That's why need multiple instance of our application.
Created 07-11-2019 01:11 PM