Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

KAFKA + RAID 10 VS JBOD on kafka

KAFKA + RAID 10 VS JBOD on kafka

hi all


we have HDP version 2.6.5

from confluent document we see that kafka maybe should works with RAID10 (https://docs.confluent.io/current/kafka/deployment.html )


but we configured the Kafka servers with multiple log directories, each directory mounted on a separate drive.

and we not have any RAID configuration ( mirrors disks )


now - lets assume we have 12 disks in the JBOD

so my question is :

in case one or two disks in kafka machine will fails

then

what is the affected on kafka machine with this situation ( dose kafka will loose data )?

Michael-Bronson
3 REPLIES 3
Highlighted

Re: KAFKA + RAID 10 VS JBOD on kafka

New Contributor

Hi Michael Bronson

To answer your question; JBOD is not recommended!

Kafka Broker will crash and stop if any disk is corrupted or any I/O Error is encountered.

Your re-assignment of data (or replicas) won't work between disks in a single broker if you want to balance the data.


Thanks,

Shesh

Re: KAFKA + RAID 10 VS JBOD on kafka

so in that case why hortonworks created the HDP 2.6.5 and kafka support JBOD in this version ?


Michael-Bronson

Re: KAFKA + RAID 10 VS JBOD on kafka

what I am saying is that - if you right , then why this option is valid in HDP 2.6.5 ?

Michael-Bronson