Hey @sharathkumar13,   Thanks for reaching out to the Cloudera community.   >> You can refer to the mentioned Git Repo[1] for information on Kafka exporter for Prometheus in Kafka Manager.   [1]https://github.com/danielqsj/kafka_exporter   >> I would like to share information on SMM[2], Streams Messaging Manager is an operations monitoring and management tool from Cloudera that provides end-to-end visibility in an enterprise Apache Kafka environment.   With SMM, you can gain clear insights about your Kafka clusters. You can understand the end-to-end flow of message streams from producers to topics to consumers.   [2]https://docs.cloudera.com/csp/2.0.1/smm-overview/topics/smm-overview.html   Let me know if this helps. ... View more

Hey @GTA,   Thanks for reaching out to the Cloudera community. "Required executor memory (1024), overhead (384 MB), and PySpark memory (0 MB) is above the max threshold (1024 MB) of this cluster!"   >> This issue occurs when the total memory required to run a spark executor in a container (Spark executor memory -> spark.executor.memory + Spark executor memory overhead: spark.yarn.executor.memoryOverhead) exceeds the memory available for running containers on the NodeManager (yarn.nodemanager.resource.memory-mb) node.   Based on the above exception you have 1 GB configured by default for a spark executor, the overhead is by default 384 MB, the total memory required to run the container is 1024+384 MB = 1408 MB.   As the NM was configured with not enough memory to even run a single container (only 1024 MB), this resulted in a valid exception.   Increasing the NM settings from 1251 to 2048 MB will definitely allow a single container to run on the NM node. Use the mentioned steps to increase "yarn.nodemanager.resource.memory-mb" parameter to resolve this.   Cloudera Manager >> YARN >> Configurations >> Search "yarn.nodemanager.resource.memory-mb" >> Configure 2048 MB or higher >> Save & Restart.   Let me know if this helps. ... View more

Hey,   Optimizing your Kafka Cluster depends upon your cluster usage & use-case.   Based on your main concern like throughput or CPU utilization or Memory/Disk usage, you need to modify different parameters and some changes may have an impact on other aspects. For example, if acknowledgments is set to "all", all brokers that replicate the partitions need to acknowledge that the data was written prior to confirming the next message needs to be sent. This will ensure data consistency but increase CPU utilization and network latency.   Refer Benchmarking Apache Kafka: 2 Million Writes Per Second (On Three Cheap Machines) article[1] written by Jay Kreps(Co-founder and CEO at Confluent).   [1]https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines   Please let me know if this helps.   Regards, Ankit. ... View more

Hey,   Can you please try setting the SPARK_HOME env variable to the location indicated by the readlink command it launches pyspark and shows Spark 2.0 as the version?   For Example: export SPARK_HOME=/opt/cloudera/parcels/SPARK2-2.3.0.cloudera3-1.cdh5.13.3.p0.458809/lib/spark2   By setting SPARK_HOME to the Spark 2 lib folder instead, pyspark2 will then launch and show Spark 2.3.0.cloudera3 as the spark version.   Please let me know if this helps.   Regards, Ankit. ... View more

Hello Wert,   As per the information provided, you have mentioned free space available.   16.09%(free: 3.2 GiB) of free space in /var/lib/cloudera-host-monitor.   35.25%(free: 4.9 GiB) of free space in /var/log/cloudera-scm-eventserver.   35.81%(free: 5.0 GiB) of free space in /var/log/cloudera-scm-alertpublisher.   Thus explaining the alert for low disk space.   The data in "/var/lib/cloudera-[host|service]-monitor" is the sum total of the working data for these respective services. Time-series metrics and health data - Time-Series Storage (firehose_time_series_storage_bytes - 10 GB default, 10 GB minimum)   My suggestions:   1.) Change the default directory("/var/lib/cloudera-[host|service]-monitor") to some other location in your environment with enough space.   >> Stop the Service(Service Monitor or Host Monitor). >> Save your old data and then copy the current directory to the new directory(optional)(Only if you need the old data). >> Update the Storage Directory configuration option (firehose.storage.base.directory) on the corresponding role configuration page. >> Start the Service Monitor or Host Monitor. 2.) If the data available in "/var/lib/cloudera-host-monitor" is not of much importance you can remove the data manually. But it's not a recommended step.   Your Health statuses will be Unknown or Bad for a short time and you will lose all Charts in the UI while the timeseries store is rebuilt and repopulated (due to the fact that you are deleting ALL the historical metrics). But this shouldn't have an impact on any service.   3.) Either add more disk to the cluster or remove unused/unnecessary files available on the disk to free up some disk space.   Regards. ... View more