Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Directory /grid/sdg/hadoop/hdfs/data became unmounted from / . Current mount point:

Solved Go to solution

Directory /grid/sdg/hadoop/hdfs/data became unmounted from / . Current mount point:

we have ambari cluster ( version 2.6 ) with 3 workers machines ( datanode machines )

each worker machine was with 5 disks ( each disk is 20G )

we upgrade the disks to 10 for each worker machine as the following:

1. on each worker we create filesystem for the new 5 disks and perform mount to the partition

2. we update the new disks on dfs.datanode.data.dir , yarn.nodemanager.local-dirs , yarn.nodemanager.log-dirs ( ambari GUI )

3. restart HDFS and YARN

but when we restart the HDFS on the workers ( datanode machines ) we get the following errors:

2017-12-31 19:01:52,509 - 
***** WARNING ***** WARNING ***** WARNING ***** WARNING ***** WARNING *****
***** WARNING ***** WARNING ***** WARNING ***** WARNING ***** WARNING *****
***** WARNING ***** WARNING ***** WARNING ***** WARNING ***** WARNING *****
Directory /grid/sdg/hadoop/hdfs/data became unmounted from / . Current mount point: /grid/sdg . 
Directory /grid/sdh/hadoop/hdfs/data became unmounted from / . Current mount point: /grid/sdh . 
Directory /grid/sdi/hadoop/hdfs/data became unmounted from / . Current mount point: /grid/sdi . 
Directory /grid/sdj/hadoop/hdfs/data became unmounted from / . Current mount point: /grid/sdj . 
Directory /grid/sdk/hadoop/hdfs/data became unmounted from / . Current mount point: /grid/sdk . Please ensure that mounts are healthy. If the mount change was intentional, you can update the contents of /var/lib/ambari-agent/data/datanode/dfs_data_dir_mount.hist.
***** WARNING ***** WARNING ***** WARNING ***** WARNING ***** WARNING *****
***** WARNING ***** WARNING ***** WARNING ***** WARNING ***** WARNING *****
***** WARNING ***** WARNING ***** WARNING ***** WARNING ***** WARNING *****


so for now all the new disks are empty without hadoop folder under /grid/sdX


please advice how to continue from this stage ?


* we restart teh ambari agent and ambari server without help


Michael-Bronson
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Directory /grid/sdg/hadoop/hdfs/data became unmounted from / . Current mount point:

need to configure the file /var/lib/ambari-agent/data/datanode/dfs_data_dir_mount.hist with all mount point , then start the data-node on each worker machine

Michael-Bronson

View solution in original post

3 REPLIES 3
Highlighted

Re: Directory /grid/sdg/hadoop/hdfs/data became unmounted from / . Current mount point:

need to configure the file /var/lib/ambari-agent/data/datanode/dfs_data_dir_mount.hist with all mount point , then start the data-node on each worker machine

Michael-Bronson

View solution in original post

Highlighted

Re: Directory /grid/sdg/hadoop/hdfs/data became unmounted from / . Current mount point:

Contributor

I have follow what @Michael Bronson indicated,
the folder /var/lib/ambari-agent/data/datanode was not existent. then i created the dfs_data_dir_mount.hist file and copied the content from another server in the cluster

the issue still not solved.

Highlighted

Re: Directory /grid/sdg/hadoop/hdfs/data became unmounted from / . Current mount point:

New Contributor

As you can see, the error message sais to check the file

/var/lib/ambari-agent/data/datanode/dfs_data_dir_mount.hist

This files stores the last mount point for each hdfs folder. In your case, seems that you are trying to mount the HDFS folders on different paths so datanode doesn't start to prevent data loss. Fix the file to point to the new mount points and start the datanode.

Don't have an account?
Coming from Hortonworks? Activate your account here