Member since
10-24-2018
4
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1510 | 11-05-2018 01:39 PM |
10-11-2019
04:30 AM
Hi, Added to the above email because if there are too many old files are available in the SHS folder the cleaner may not work as expected. So the ideal way to be delete manually if there are too old .inprogress files. Thanks AKR
... View more
01-29-2019
02:49 PM
Go to ResourceManager UI on Ambari. Click
nodes link on the left side of the window. It should show all Node
Managers and the reason for it being listed as unhealthy. Mostly found reasons are regarding disk space threshold
reached. In that case needs to consider following parameters
Parameters
Default value
Description
yarn.nodemanager.disk-health-checker.min-healthy-disks
0.25
The minimum fraction of number of disks to be healthy for the
node manager to launch new containers. This correspond to both
yarn.nodemanager.local-dirs and yarn.nodemanager.log-dirs. i.e. If there are
less number of healthy local-dirs (or log-dirs) available, then new
containers will not be launched on this node.
yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage
90.0
The maximum percentage of disk space utilization allowed after
which a disk is marked as bad. Values can range from 0.0 to 100.0. If the
value is greater than or equal to 100, the nodemanager will check for full
disk. This applies to yarn.nodemanager.local-dirs and
yarn.nodemanager.log-dirs.
yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb
0
The minimum space that must be available on a disk for it to
be used. This applies to yarn.nodemanager.local-dirs and
yarn.nodemanager.log-dirs.
In the final step, if above steps do not reveal the actual
problem , needs to check log , location path : /var/log/hadoop-yarn/yarn.
... View more
11-05-2018
01:39 PM
We are able to resolve the issue by applying below
configurations on the Oozie action in xml. <name>mapreduce.job.user.classpath.first</name> <value>true</value> </property> <property> <name>mapreduce.task.classpath.user.precedence</name> <value>true</value> </property> <property> <name>oozie.launcher.mapreduce.task.classpath.user.precedence</name> <value>true</value> </property> <property> <name>oozie.launcher.mapreduce.job.user.classpath.first</name> <value>true</value> </property> To add,
we got the reference of these properties from a related previous post. See
below link. https://community.hortonworks.com/questions/114525/oozie-overrides-dependencies-with-shared-libsprobl.html
... View more