Member since
08-08-2017
1652
Posts
30
Kudos Received
11
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2093 | 06-15-2020 05:23 AM | |
| 17460 | 01-30-2020 08:04 PM | |
| 2256 | 07-07-2019 09:06 PM | |
| 8729 | 01-27-2018 10:17 PM | |
| 4914 | 12-31-2017 10:12 PM |
05-31-2018
10:12 AM
dear friends and colleges we set the KAFKA_JVM_PERFORMANCE_OPTS on all ours kafka physical machines , according to the following Link: https://community.hortonworks.com/articles/80813/kafka-best-practices-1.html we set in ambari - Advanced kafka-env the following: export KAFKA_JVM_PERFORMANCE_OPTS="-XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 -XX:MaxMetaspaceFreeRatio=80" after we set that variable we restart the kafka broker to take affect but kafka broker fails after this settings , please advice what is wrong on the settings from the article - https://community.hortonworks.com/articles/80813/kafka-best-practices-1.html from the logs we can see that: from kafka.err more kafka.err
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Unrecognized VM option 'UseG1GC-XX:MaxGCPauseMillis=20'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Unrecognized VM option 'UseG1GC-XX:MaxGCPauseMillis=20'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Unrecognized VM option 'UseG1GC-XX:MaxGCPauseMillis=20'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Unrecognized VM option 'UseG1GC-XX:MaxGCPauseMillis=20'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit. from server.log [2018-05-31 09:03:19,334] INFO starting (kafka.server.KafkaServer)
[2018-05-31 09:03:19,350] INFO [ThrottledRequestReaper-Fetch], Starting (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2018-05-31 09:03:19,351] INFO [ThrottledRequestReaper-Produce], Starting (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2018-05-31 09:03:19,358] INFO Connecting to zookeeper on master01.sys674.com:2181,master02.sys674.com:2181,master03.sys674.com:2181 (kafka.server.KafkaServer)
[2018-05-31 09:03:19,639] INFO Cluster ID = 7BUsZLW1RJa54GJ0CDCtcg (kafka.server.KafkaServer)
[2018-05-31 09:03:19,682] INFO Loading logs. (kafka.log.LogManager)
[2018-05-31 09:03:19,690] INFO Logs loading complete in 8 ms. (kafka.log.LogManager)
[2018-05-31 09:03:19,848] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2018-05-31 09:03:19,850] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2018-05-31 09:03:19,855] WARN No meta.properties file under dir /var/kafka/kafka-logs/meta.properties (kafka.server.BrokerMetadataCheckpoint)
[2018-05-31 09:03:19,913] INFO Awaiting socket connections on kafka01.sys674.com:6667. (kafka.network.Acceptor)
[2018-05-31 09:03:19,917] INFO [Socket Server on Broker 1003], Started 1 acceptor threads (kafka.network.SocketServer)
[2018-05-31 09:03:19,943] INFO [ExpirationReaper-1003], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:03:19,944] INFO [ExpirationReaper-1003], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:03:19,991] INFO [ExpirationReaper-1003], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:03:19,994] INFO [ExpirationReaper-1003], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:03:19,995] INFO [ExpirationReaper-1003], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:03:20,077] INFO [GroupCoordinator 1003]: Starting up. (kafka.coordinator.GroupCoordinator)
[2018-05-31 09:03:20,081] INFO [GroupCoordinator 1003]: Startup complete. (kafka.coordinator.GroupCoordinator)
[2018-05-31 09:03:20,082] INFO [Group Metadata Manager on Broker 1003]: Removed 0 expired offsets in 1 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2018-05-31 09:03:20,118] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2018-05-31 09:03:20,172] INFO Creating /brokers/ids/1003 (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2018-05-31 09:03:20,187] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2018-05-31 09:03:20,189] INFO Registered broker 1003 at path /brokers/ids/1003 with addresses: PLAINTEXT -> EndPoint(kafka01.sys674.com,6667,PLAINTEXT) (kafka.utils.ZkUtils)
[2018-05-31 09:03:20,190] WARN No meta.properties file under dir /var/kafka/kafka-logs/meta.properties (kafka.server.BrokerMetadataCheckpoint)
[2018-05-31 09:03:20,273] INFO [Kafka Server 1003], started (kafka.server.KafkaServer)
[2018-05-31 09:13:20,078] INFO [Group Metadata Manager on Broker 1003]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2018-05-31 09:23:20,078] INFO [Group Metadata Manager on Broker 1003]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2018-05-31 09:33:20,078] INFO [Group Metadata Manager on Broker 1003]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2018-05-31 09:43:20,078] INFO [Group Metadata Manager on Broker 1003]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2018-05-31 09:46:25,337] INFO [Kafka Server 1003], shutting down (kafka.server.KafkaServer)
[2018-05-31 09:46:25,339] INFO [Kafka Server 1003], Starting controlled shutdown (kafka.server.KafkaServer)
[2018-05-31 09:46:25,393] INFO [Kafka Server 1003], Controlled shutdown succeeded (kafka.server.KafkaServer)
[2018-05-31 09:46:25,396] INFO [Socket Server on Broker 1003], Shutting down (kafka.network.SocketServer)
[2018-05-31 09:46:25,406] INFO [Socket Server on Broker 1003], Shutdown completed (kafka.network.SocketServer)
[2018-05-31 09:46:25,407] INFO [Kafka Request Handler on Broker 1003], shutting down (kafka.server.KafkaRequestHandlerPool)
[2018-05-31 09:46:25,413] INFO [Kafka Request Handler on Broker 1003], shut down completely (kafka.server.KafkaRequestHandlerPool)
[2018-05-31 09:46:25,420] INFO [ThrottledRequestReaper-Fetch], Shutting down (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2018-05-31 09:46:25,595] INFO [ThrottledRequestReaper-Fetch], Stopped (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2018-05-31 09:46:25,595] INFO [ThrottledRequestReaper-Fetch], Shutdown completed (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2018-05-31 09:46:25,595] INFO [ThrottledRequestReaper-Produce], Shutting down (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2018-05-31 09:46:25,621] INFO [ThrottledRequestReaper-Produce], Stopped (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2018-05-31 09:46:25,621] INFO [ThrottledRequestReaper-Produce], Shutdown completed (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2018-05-31 09:46:25,622] INFO [KafkaApi-1003] Shutdown complete. (kafka.server.KafkaApis)
[2018-05-31 09:46:25,624] INFO [Replica Manager on Broker 1003]: Shutting down (kafka.server.ReplicaManager)
[2018-05-31 09:46:25,624] INFO [ReplicaFetcherManager on broker 1003] shutting down (kafka.server.ReplicaFetcherManager)
[2018-05-31 09:46:25,626] INFO [ReplicaFetcherManager on broker 1003] shutdown completed (kafka.server.ReplicaFetcherManager)
[2018-05-31 09:46:25,626] INFO [ExpirationReaper-1003], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:25,671] INFO [ExpirationReaper-1003], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:25,671] INFO [ExpirationReaper-1003], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:25,671] INFO [ExpirationReaper-1003], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:25,832] INFO [ExpirationReaper-1003], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:25,832] INFO [ExpirationReaper-1003], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:25,835] INFO [Replica Manager on Broker 1003]: Shut down completely (kafka.server.ReplicaManager)
[2018-05-31 09:46:25,835] INFO [ExpirationReaper-1003], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:25,873] INFO [ExpirationReaper-1003], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:25,873] INFO [ExpirationReaper-1003], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:25,874] INFO [GroupCoordinator 1003]: Shutting down. (kafka.coordinator.GroupCoordinator)
[2018-05-31 09:46:25,874] INFO [ExpirationReaper-1003], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:26,067] INFO [ExpirationReaper-1003], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:26,067] INFO [ExpirationReaper-1003], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:26,067] INFO [ExpirationReaper-1003], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:26,125] INFO [ExpirationReaper-1003], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:26,125] INFO [ExpirationReaper-1003], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2018-05-31 09:46:26,126] INFO [GroupCoordinator 1003]: Shutdown complete. (kafka.coordinator.GroupCoordinator)
[2018-05-31 09:46:26,127] INFO Shutting down. (kafka.log.LogManager)
[2018-05-31 09:46:26,138] INFO Shutdown complete. (kafka.log.LogManager)
[2018-05-31 09:46:26,151] INFO [Kafka Server 1003], shut down completed (kafka.server.KafkaServer)
<br>
... View more
Labels:
05-28-2018
07:22 AM
( I accept your anser but we have aproblem ) , regarding the - version"+str(int(time.time() * 1000000)) , what in case its return number that already exists as version ( in spite this is very very rare ? )
... View more
05-27-2018
03:08 PM
we want to post the new config to ambari but we need uniq tag number that not exist in ambari how to generate new tag number by API or other approach , so we can put it after "tag" ? as - 237432322 curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '[
{
"Clusters": {
"desired_config": [
{
"type": "kafka-env",
"tag": "237432322",
"properties": {
... View more
Labels:
05-27-2018
09:40 AM
we have hadoop cluster version 2.6.4 with 3 kafka machines we are thinking what is the best filesystem for the kafka disk - sdb ( 25T ) according to the article - https://community.hortonworks.com/articles/80813/kafka-best-practices-1.html "FileSystem Selection Kafka uses regular files on disk, and such it has no hard dependency on a specific file system. We recommend EXT4 or XFS. Recent improvements to the XFS file system have shown it to have the better performance characteristics for Kafka’s workload without any compromise in stability. Note: Do not use mounted shared drives and any network file systems. In our experience Kafka is known to have index failures on such file systems. Kafka uses MemoryMapped files to store the offset index which has known issues on anetwork file systems." seems that they recommended on xfs file system and not ext 4 but xfs is old file system , so I am little confuse I will happy to know more opiones on that my sdb ( kafka disk ) df | grep sdb NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb 8:16 0 25.5T 0 disk
... View more
Labels:
05-25-2018
01:29 PM
example of final file <br>{
"href" : "http://master02:8080/api/v1/clusters/HDP/configurations?type=kafka-env&tag=version1527250007610",
"items" : [
{
"href" : "http://master02:8080/api/v1/clusters/HDP26/configurations?type=kafka-env&tag=version1527250007610",
"tag" : "version1527250007610",
"type" : "kafka-env",
"version" : 8,
"Config" : {
"cluster_name" : "HDP",
"stack_id" : "HDP-2.6"
},
"properties" : {
"content" : "\n#!/bin/bash\n\n# Set KAFKA specific environment variables here.\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}\nexport PATH=$PATH:$JAVA_HOME/bin\nexport PID_DIR={{kafka_pid_dir}}\nexport LOG_DIR={{kafka_log_dir}}\nexport KAFKA_KERBEROS_PARAMS={{kafka_kerberos_params}}\nexport JMX_PORT=9997\n# Add kafka sink to classpath and related depenencies\nif [ -e \"/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar\" ]; then\n export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar\n export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/lib/*\nfi\n\nif [ -f /etc/kafka/conf/kafka-ranger-env.sh ]; then\n. /etc/kafka/conf/kafka-ranger-env.sh\nfi\nexport KAFKA_HEAP_OPTS=\"-Xmx8g -Xms8g\"\nKAFKA_JVM_PERFORMANCE_OPTS=\"-XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80\n"",
"is_supported_kafka_ranger" : "true",
"kafka_log_dir" : "/var/log/kafka",
"kafka_pid_dir" : "/var/run/kafka",
"kafka_user" : "kafka",
"kafka_user_nofile_limit" : "128000",
"kafka_user_nproc_limit" : "65536"
}
}
]
... View more
05-25-2018
12:38 PM
so in that case I prefer to separate the variables as the following: export KAFKA_HEAP_OPTS="-Xmx8g -Xms8g"
AND
export KAFKA_JVM_PERFORMANCE_OPTS="-XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80" can you please confirm that? so both them will be added to kafka-env
... View more
05-25-2018
11:17 AM
so the steps in https://community.hortonworks.com/questions/193769/how-to-add-variable-in-kafka-env-template-by-api.html are the same steps for ? export KAFKA_HEAP_OPTS="-Xmx8g -Xms8g -XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80"
... View more
05-25-2018
11:03 AM
as export KAFKA_HEAP_OPTS="-Xmx8g -Xms8g -XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80" ?
... View more
05-25-2018
11:02 AM
Or in simple words can we set all "-Xmx8g -Xms8g -XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80" in KAFKA_HEAP_OPTS ?
... View more