Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS Non DFS Used (History server)

HDFS Non DFS Used (History server)

New Contributor

HDP-2.3.4.0-3485

HDFS version: 2.7.1.2.3

<Reproduce Steps>

1. su hdfs

2. cd /home/hdfs

3. hadoop jar tkfc-static-wordlist-10.jar 1 issue money 20150601 (extracting data elements from webpages)

<Actual Result>

1. usage non used DFS Continued to increase

2. HDFS service will shut down

2 REPLIES 2

Re: HDFS Non DFS Used (History server)

You can increase space for non-HDFS use by setting dfs.datanode.du.reserved to a larger value. It's set per disk volume. The default is 1G, but we usually set it to 10G. You can increase it further depending on number of your disks and their capacity. However, non HDFS space is not used by History Server but to store for example output of mappers in a MR job.

Re: HDFS Non DFS Used (History server)

New Contributor

This approach seems only delay occurred at a time, after the release of this issue will be resolved?

jira reference

https://issues.apache.org/jira/browse/HDFS-3570?jql=text%20~%20%22Non%20DFS%22 https://issues.apache.org/jira/browse/HDFS-9038

When will the update to hdp?

Don't have an account?
Coming from Hortonworks? Activate your account here