Member since
09-18-2018
7
Posts
0
Kudos Received
0
Solutions
02-12-2019
09:50 PM
Hi i am using /usr/java/jdk1.8.0_202 and i am facing issue even if i submit the export JAVA_HOME=/usr/java/jdk1.8.0_202 command. i am stucked in the middle of a job with the following error. please help me to sort out the issue. thanks in advance
... View more
02-04-2019
10:37 PM
Diskuse=$(hdfs dfs -du /user |sort -n -r | awk '{print $1/1099511531398.98," GB", $2/1099511531398.98," TB ", $3}'| head -10) Diskuse=$(hdfs dfs -du /user |sort -n -r | awk '{print $1/1073741824," GB", $2/1073741824," TB ", $3}'| head -10)
... View more
01-30-2019
01:47 AM
Hi,
I am trying to automate the HDFS user space utilisation mail.
Everything went correct exept the sorting of the files in gb.when i tried with bytes it is giving the correct result but in other it is not coming the expected output.
please help me the get the correct output.
i ll provide the script which i run in bytes as well as human readable -h in hdfs file system.
#!/bin/bash
#getting the current hdfs percentage in numeric value
CURRENT=$(hdfs dfs -df -h/ | grep / | awk '{ print $8}' | sed 's/%//g')
#current hdfs space utilisation DiskFile=$(hdfs dfs -df -h)
HdfsReport=$(hdfs dfsadmin -report)
Diskuse=$(hdfs dfs -du /user | sort -nr | head -10)
#To get results GB i have provided $(hdfs dfs -du -h /user | sort -r | head -10) THRESHOLD=70 Critical=90
if [ "$CURRENT" -gt "$THRESHOLD" ] ; then
mail -s 'HDFS Usage Housekeeping required' @ABC.com, @ABC.com << EOF HDFS usage in Cluster is above the threshold please run the clean-up scripts asap. Used: $CURRENT%
Current disk utilization report is $DiskFile
Please find the Utilisation report of top ten users consuming the cluster
$Diskuse
EOF fi
if [ "$CURRENT" -gt "$Critical" ] ; then
mail -s 'HDFS Admin Report' yy@abc.com, yyy@abc.com << EOF HDFS usage in Cluster is above critical storage, please Find the Cluster report below
$HdfsReport
EOF fi
Please help me to sort out the this.
Thanks in advance
... View more
Labels:
- Labels:
-
HDFS