In order to use Hadoop 2.6 storage policies you must specify the type for each mount point in dfs.datanode.data.dir the type (DISK, ARCHIVE, RAM_DISK, SSD). If editing the hdfs-site.xml file directly I would do -
<property> <name>dfs.datanode.data.dir</name> <value>[ARCHIVE]file:///mnt/archive/dfs/dn,[SSD]file:///mnt/flash/dfs/dn,[DISK]file:///mnt/disk/dfs/dn</value> </property>
However, if I try and use this format in the CM GUI I get the following error -
Does anyone know what is the correct format to specify storage tiers in the GUI or how to manually bypass the GUI and configure this.
Thanks - that worked.
I'm assuming that I should leave the mounts in the dfs.datanode.data.dir section, so that CM knows to monitor the mounts.
Some more questions based on this thread
Once storage configuration is defined and SSDs/ Disks are identified by HDFS,