Hi Chethan, Thank you for the reply. The parcel version was selected automatically after we clicked 'download' in the Cloudera Manager. I suppose it is the correct version.    2022-05-24 11:04:57,774 INFO New I/O worker #340:com.cloudera.parcel.components.ParcelDownloaderImpl: Completed download of: https://archive.cloudera.com/kafka/parcels/latest/KAFKA-4.1.0-1.4.1.0.p0.4-el7.parcel   However from this page, it shows for CDH 6.3.4, the Apache Kafka version is 2.2.1 https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_63_packaging.html   Not sure how the parcel version and Apache version relate to each other. We have the complete log file, but I hesitate to send the log here in an open forum. Is there a way to send to you in a protected mode? Thanks again! ... View more

@vincentD I hope it helped to resolve your issue, Regarding your query on upgrading CDH 6.3.4 to new software version.  Please explore the below CDP Path.  https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/release-guide/topics/cdpdc-release-notes-links.html https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/release-guide/topics/cm-cdh-runtime-versions.html https://docs.cloudera.com/cdp-private-cloud/latest/release-summaries/topics/announcement-202108-717.html https://docs.cloudera.com/cdp-private-cloud/latest/index.html Cheers! Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button. ... View more

Thanks very much @MyNamesNotRick . We will check them out.  ... View more

Hello @vincentD Please review the stdout and stderr of the DN which going down frequently. You can navigate to CM > HDFS > Instance > Select the DN which went down > Processes > click on stdout/stderr atthe bottom of the page.     I am asking to verify stdout/stderr suspecting an OOM error (due to java heap running out of memory) leading to the DN exit/shutdown abruptly.     If the DN exit is due to OOM Error, please increase the DN heap size to adequate value to get rid off teh issue further. DN heap sizing rule of thumb says: 1 GB heap memory for 1Million blocks. You can verify your block counts on each DN by navigating to CM > HDFS > NN Web UI > Active NN > DataNode and you can see the DN stats on that page showing block counts and disk usage etc.. ... View more

Hi paras, you are very helpful. The error disappears now after making the configuration modification as you suggested. Thank you! ... View more

Depends on what you mean by 'storage locations'. If you mean "can other apps use HDFS?" then the answer is yes, as HDFS is an independent system unrelated to YARN and has its own access and control mechanisms not governed by a YARN scheduler. If you mean "can other apps use the scratch space on NM nodes" then the answer is no, as only local containers get to use that. If you're looking to strictly split both storage and compute, as opposed to just some form of compute, then it may be better to divide up the cluster entirely. ... View more