we have Hadoop cluster ( version 2.6.4 ) with 3 physical kafka machines we want to know what is the
values of Xmx and Xms that we need to allocate on the kafka machines in order to setup the in
the Xmx and Xms on kafka machine we need to configure the
script /usr/hdp/2.6.4 /kafka/bin/kafka-server-start for now the default values are ( Xmx is 1G and Xms is 1G ) if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
export
KAFKA_HEAP_OPTS="-Xmx1G –Xms1G"
fi
[root@kafka01 ~]# free -g
total used free shared buff/cache available
Mem: 251 17 11 0 222 233
Swap: 15 6 9
on each kafka we have 256G and we found this link - https://stackoverflow.com/questions/4667483/how-is-the-default-java-heap-size-determined according to the link in
stackoverflow we can use this formula: [root@kafka01 ~]# java -XX:+PrintFlagsFinal -version | grep HeapSize
uintx ErgoHeapSizeLimit = 0 {product}
uintx HeapSizePerGCThread = 87241520 {product}
uintx InitialHeapSize := 2147483648 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 32210157568 {product}
openjdk version "1.8.0_65"
OpenJDK Runtime Environment (build 1.8.0_65-b17)
OpenJDK 64-Bit Server VM (build 25.65-b01, mixed mode)
[root@kafka01 ~]# java -XX:+PrintFlagsFinal -version | grep HeapSize
uintx ErgoHeapSizeLimit = 0 {product}
uintx HeapSizePerGCThread = 87241520 {product}
uintx InitialHeapSize := 2147483648 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 32210157568 {product}
openjdk version "1.8.0_65"
OpenJDK Runtime Environment (build 1.8.0_65-b17)
OpenJDK 64-Bit Server VM (build 25.65-b01, mixed mode) So the values comes with bytes
and we can calculate them to Giga So Xmx = 32G Xms = 2G What is the recommendation of
hortonworks about Xmx and Xms ? Dose hortonworks except the
formula - java -XX:+PrintFlagsFinal -version | grep HeapSize In order to get the right values
of Xmx and Xms ?
... View more
