Member since
02-17-2019
30
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1697 | 05-19-2020 08:47 AM |
05-31-2022
07:35 AM
Hi Chethan, Thank you for the reply. The parcel version was selected automatically after we clicked 'download' in the Cloudera Manager. I suppose it is the correct version. 2022-05-24 11:04:57,774 INFO New I/O worker #340:com.cloudera.parcel.components.ParcelDownloaderImpl: Completed download of: https://archive.cloudera.com/kafka/parcels/latest/KAFKA-4.1.0-1.4.1.0.p0.4-el7.parcel However from this page, it shows for CDH 6.3.4, the Apache Kafka version is 2.2.1 https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_63_packaging.html Not sure how the parcel version and Apache version relate to each other. We have the complete log file, but I hesitate to send the log here in an open forum. Is there a way to send to you in a protected mode? Thanks again!
... View more
10-06-2021
08:34 AM
@vincentD I hope it helped to resolve your issue, Regarding your query on upgrading CDH 6.3.4 to new software version. Please explore the below CDP Path. https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/release-guide/topics/cdpdc-release-notes-links.html https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/release-guide/topics/cm-cdh-runtime-versions.html https://docs.cloudera.com/cdp-private-cloud/latest/release-summaries/topics/announcement-202108-717.html https://docs.cloudera.com/cdp-private-cloud/latest/index.html Cheers! Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
09-30-2020
10:14 PM
Hello @vincentD Please review the stdout and stderr of the DN which going down frequently. You can navigate to CM > HDFS > Instance > Select the DN which went down > Processes > click on stdout/stderr atthe bottom of the page. I am asking to verify stdout/stderr suspecting an OOM error (due to java heap running out of memory) leading to the DN exit/shutdown abruptly. If the DN exit is due to OOM Error, please increase the DN heap size to adequate value to get rid off teh issue further. DN heap sizing rule of thumb says: 1 GB heap memory for 1Million blocks. You can verify your block counts on each DN by navigating to CM > HDFS > NN Web UI > Active NN > DataNode and you can see the DN stats on that page showing block counts and disk usage etc..
... View more
06-03-2020
07:08 AM
Hi paras, you are very helpful. The error disappears now after making the configuration modification as you suggested. Thank you!
... View more
05-19-2020
08:47 AM
Hi paras, resetting the hostnames to long names got me moving forward again. Thank you!
... View more
05-23-2019
08:46 PM
1 Kudo
For HBase MOBs, this can serve as a good starting point as most of the changes are administrative and the writer API remains the same as regular cells: https://www.cloudera.com/documentation/enterprise/latest/topics/admin_hbase_mob.html For SequenceFiles, a good short snippet can be found here: https://github.com/sakserv/sequencefile-examples/blob/master/test/main/java/com/github/sakserv/sequencefile/SequenceFileTest.java#L65-L70 and for Parquet: https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/example/ExampleParquetWriter.java More general reading for the file formats: https://blog.cloudera.com/blog/2011/01/hadoop-io-sequence-map-set-array-bloommap-files/ and https://parquet.apache.org/documentation/latest/
... View more
05-07-2019
09:58 PM
Depends on what you mean by 'storage locations'. If you mean "can other apps use HDFS?" then the answer is yes, as HDFS is an independent system unrelated to YARN and has its own access and control mechanisms not governed by a YARN scheduler. If you mean "can other apps use the scratch space on NM nodes" then the answer is no, as only local containers get to use that. If you're looking to strictly split both storage and compute, as opposed to just some form of compute, then it may be better to divide up the cluster entirely.
... View more