@Balkrishna, The problem is with special characters in the KAFKA service metrics file. We use this file as a part of stack_advisor calculations for AMS split points. Following is a grep on non ASCII characters that reveals the problem :- grep --color='auto' -P -n "[\x80-\xFF]" /var/lib/ambari-server/resources/common-services/AMBARI_METRICS/0.1.0/package/files/service-metrics/KAFKA.txt This will show you the ascii characters present in the file as, Output:-43:��kafka.network.RequestMetrics.RequestsPerSec.request.OffsetFetch.count��
45:��kafka.network.RequestMetrics.RequestsPerSec.request.OffsetCommit.count
47:kafka.network.RequestMetrics.RequestsPerSec.request.LeaderAndIsr.1MinuteRate�� Use /var/lib/ambari-server/resources/scripts/configs.sh to modify and get values from ambari-server as, /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p ADMIN_PASSWORD -port 8080 get localhost CLUSTER_NAME ams-site Check the value of "timeline.metrics.cluster.aggregate.splitpoints" and "timeline.metrics.host.aggregate.splitpoints", Look for special non-ascii characters example: "dfs.datanode.ReplaceBlockOpAvgTime,kafka.network.RequestMetrics.RequestsPerSec.request.JoinGroup.1MinuteRate ,master.Master.ProcessCallTime_num_ops,regionserver.Server.blockCacheEvictionCount" Here after 1MinuteRate there is space which will shown as a special character on browser through API call as, "http://AMBARI_SERVER_HOSTS:8080/api/v1/clusters/CLUSTER_NAME/configurations?type=ams-site" Take latest tag from it at the bottom of page. and put it on browser you will see the special character as, "timeline.metrics.cluster.aggregate.splitpoints":"dfs.datanode.ReplaceBlockOpAvgTime,kafka.network.RequestMetrics.RequestsPerSec.request.JoinGroup.1MinuteRate
,master.Master.ProcessCallTime_num_ops,regionserver.Server.blockCacheEvictionCount" To resolve this issue use, /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p ADMIN_PASSWORD -port 8080 set localhost CLUSTER_NAME ams-site timeline.metrics.cluster.aggregate.splitpoints dfs.datanode.ReplaceBlockOpAvgTime,kafka.network.RequestMetrics.RequestsPerSec.request.JoinGroup.1MinuteRate,master.Master.ProcessCallTime_num_ops,regionserver.Server.blockCacheEvictionCount To change second property paramater use:- /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p ADMIN_PASSWORD -port 8080 get localhost CLUSTER_NAME ams-site /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p ADMIN_PASSWORD -port 8080 set localhost CLUSTER_NAME ams-site timeline.metrics.host.aggregate.splitpoints EventTakeSuccessCount,cpu_idle,dfs.FSNamesystem.ExcessBlocks,dfs.datanode.ReadBlockOpNumOps,disk_total,jvm.JvmMetrics.LogError,kafka.controller.ControllerStats.LeaderElectionRateAndTimeMs.99percentile,kafka.network.RequestMetrics.RequestsPerSec.request.FetchFollower.5MinuteRate,kafka.network.RequestMetrics.RequestsPerSec.request.UpdateMetadata.1MinuteRate,kafka.server.BrokerTopicMetrics.FailedFetchRequestsPerSec.meanRate,master.AssignmentManger.ritCount,master.FileSystem.MetaHlogSplitTime_95th_percentile,mem_shared,proc_total,regionserver.Server.Append_median,regionserver.Server.Replay_95th_percentile,regionserver.Server.totalRequestCount,rpcdetailed.rpcdetailed.GetBlockLocationsAvgTime,write_bps
### Restart the Service
... View more
