Member since
11-12-2018
218
Posts
179
Kudos Received
35
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
349 | 08-08-2025 04:22 PM | |
423 | 07-11-2025 08:48 PM | |
649 | 07-09-2025 09:33 PM | |
1132 | 04-26-2024 02:20 AM | |
1490 | 04-18-2024 12:35 PM |
11-24-2019
09:52 PM
Hi @anshuman Yes, we have Node labels support in the new CDP. For more details, you can check CDP documentations. https://docs.cloudera.com/ -> Cloudera Data Platform -> Runtime -> Cloudera Runtime https://docs.cloudera.com/runtime/7.0.2/yarn-allocate-resources/topics/yarn-configuring-node-labels.html FYI. Cloudera Runtime is the core open-source software distribution within Cloudera Data Platform (CDP) that is maintained, supported, versioned, and packaged as a single entity by Cloudera. Cloudera Runtime includes approximately 50 open source projects that comprise the core distribution of data management tools within CDP, including Cloudera Manager, which is used to configure and monitor clusters managed in CDP.
... View more
01-21-2019
12:37 PM
@Jay , after we do the decommission on some datanode , do we need also to stop the components on that datanode ? and then replaced the disk , or maybe it is inoufgh to decomission without to stop the componet and then replaced the disk ?
... View more
01-02-2019
07:19 AM
hi Geoffrey , I post anew thread - https://community.hortonworks.com/questions/231177/metrics-failed-on-orgapachehadoophbasezookeeperzoo.html , but I see this post not appears in the hortonworks questions ,could you help me to understand why ?
... View more
01-02-2019
11:18 AM
1 Kudo
@Suraj Singh Seems it's similar like what we discussed in below thread https://community.hortonworks.com/questions/232093/yarn-jobs-are-getting-stuck-in-accepted-state.html If resubmit jobs will get success ? As discussed earlier this is open bug which fixed in further releases. If you need to apply patch, please involve Hortonworks support. If you are a customer, HWX can release a patch for you if it's technically possible based on specifics of the JIRAs. If you don't have support, you can certainly do it but test it first apply the patch in dev/test and see if it resolves your problem.
... View more
12-28-2018
03:31 PM
2 Kudos
@Slimani Ibrahim
It's not recommended to format the NameNode more than once except when NameNode loses metadata information. The reason could be this property which tells NameNode where to store its metadata information on disk is dfs.namenode.name.dir in your case its points to /tmp, so every time you restart your system the /tmp directory gets flushed and hence you have to format the NameNode again. So, make sure you point the property dfs.namenode.name.dir to a more persistent location (something like /hadoop/hdfs/namenode similar for datanode property/hadoop/hdfs/datanode) which does not get's cleared every time if you restart your system that will resolve this problem. I hope that the above answers your questions. Please accept the answer you found most useful.
... View more
01-01-2019
10:31 AM
2 Kudos
@Suraj Singh Actually particular fix is released in following version 3.1.0, 2.10.0, 2.9.1, 3.0.1 . Related JIRA https://issues.apache.org/jira/browse/YARN-7873. Fix you read complete comments then you will get idea about why they revert YARN-6078 and not released on 3.0.0.
... View more
12-27-2018
10:57 AM
3 Kudos
@Michael Bronson After configuration changes, it's safe to restart required services, those restart will make necessary new changes into the system. In our case, yarn.nodemanager.local-dirs will point out to new location /grid/sdb/hadoop/yarn/local instead of old location /var/hadoop/yarn/local . In short, restart will not cause any issue either after delete old files or after change in YARN configuration. I hope this answered your concerns.
... View more
12-27-2018
12:16 PM
Sorting is carried out at the Map side. When all the map outputs have been copied, the reduce task moves into the sort phase i.e.maerging phase. which merges the map outputs, maintaining their sort ordering. This is done in rounds. For example, if there were 60 map outputs and the merge factor was 15 (the default, controlled by the mapreduce.task.io.sort.factor property, just like in the map’s merge), there would be four rounds. Each round would merge 15 files into 1, so at the end, there would be 4 intermediate files to be processed. This is done using a key-value pair.
... View more
12-26-2018
10:21 AM
2 Kudos
@Anjali Shevadkar Below article help you to configure spark fine grain security. https://community.hortonworks.com/content/kbentry/72454/apache-spark-fine-grain-security-with-llap-test-dr.html Please accept the answer you found most useful.
... View more