Why am I getting disk usage alert?

[#1]

I created 7 AWS EC2 instances (1 for ambari server and 6 for ambari agents) and installed HDP 2.4 using ambari. Now, when I open the ambari dashboard, it shows me critical alerts related to disk usage. It shows that the hosts have less than 6GB disk size. For example, the first host in the attached image shows "4.56 GB/5.63 (88.1% used)". How can I solve this problem?

[#2]

The metrics for the first host in the attached image is shown below.

[#3]

Can you get the o/p of command :

$hdfs dfsadmin -report

This will get us the actual state of disks. Also, we can then figure out if the alerts are stale.

- Also, if we are running out of disk, if would be good idea to increase your disk space.

[#4]

I am getting the report below in the hosts

[#5]

@Fish Berh

Missed on mentioning 'su hdfs', to get the full report:

Can you once again do:

$su hdfs

$hdfs dfsadmin -report

[#6]

When I use the command $su hdfs, it asks me for password. However, I have not set any password.

[#7]

@Fish Berh Did you "su hdfs" as root/admin user ?

[#8]

I changed to root using "sudo su -", then ran the command "su hdfs", but I get an error message that user "hdfs" is not known.

[#9]

Hi @Fish Berh

The default alert configuration for disk_usage is set like the one below:

I believe the minimum free space is kicking in here.

 "AlertDefinition" : {

    "cluster_name" : "xxxxxxxxxx",

    "component_name" : "AMBARI_AGENT",

    "description" : "This host-level alert is triggered if the amount of disk space used goes above specific thresholds. The default threshold values are 50% for WARNING and 80% for CRITICAL",

    "enabled" : true,

    "id" : 48,

    "ignore_host" : false,

    "interval" : 1,

    "label" : "Host Disk Usage",

    "name" : "ambari_agent_disk_usage",

    "scope" : "HOST",

    "service_name" : "AMBARI",

    "source" : {

      "parameters" : [

        {

          "name" : "minimum.free.space",

          "description" : "The overall amount of free disk space left before an alert is triggered.",

          "threshold" : "WARNING",

          "units" : "bytes",

          "display_name" : "Minimum Free Space",

          "type" : "NUMERIC",

          "value" : "5.0E9"

        },

        {

          "name" : "percent.used.space.warning.threshold",

          "description" : "The percent of disk space consumed before a warning is triggered.",

          "threshold" : "WARNING",

          "units" : "%",

          "display_name" : "Warning",

          "type" : "PERCENT",

          "value" : “0.5”

        },

        {

          "name" : "percent.free.space.critical.threshold",

          "description" : "The percent of disk space consumed before a critical alert is triggered.",

          "threshold" : "CRITICAL",

          "units" : "%",

          "display_name" : "Critical",

          "type" : "PERCENT",

          "value" : “0.8”

        }

      ],

      "path" : "alert_disk_space.py",

      "type" : "SCRIPT"

    }

  }

}

Hope this helps.

[#10]

But it is saying it has used 88.1% of 5.63 GB. But when I check the disk size in the terminals, I the report shown in the attachment