Member since
06-24-2015
2
Posts
0
Kudos Received
0
Solutions
06-25-2015
01:14 PM
Thanks - that worked. I'm assuming that I should leave the mounts in the dfs.datanode.data.dir section, so that CM knows to monitor the mounts.
... View more
06-24-2015
05:18 AM
In order to use Hadoop 2.6 storage policies you must specify the type for each mount point in dfs.datanode.data.dir the type (DISK, ARCHIVE, RAM_DISK, SSD). If editing the hdfs-site.xml file directly I would do - <property>
<name>dfs.datanode.data.dir</name>
<value>[ARCHIVE]file:///mnt/archive/dfs/dn,[SSD]file:///mnt/flash/dfs/dn,[DISK]file:///mnt/disk/dfs/dn</value>
</property> However, if I try and use this format in the CM GUI I get the following error - DataNode Data Directory: Path [ARCHIVE]:///mnt/archive/dfs/dn does not conform to the pattern "(/[-+=_.a-zA-Z0-9]+)+(/)*" DataNode Data Directory: Path [DISK]:file///mnt/disk/dfs/dn does not conform to the pattern "(/[-+=_.a-zA-Z0-9]+)+(/)*" DataNode Data Directory: Path [SSD]:file///mnt/flash/dfs/dn does not conform to the pattern "(/[-+=_.a-zA-Z0-9]+)+(/)*" Does anyone know what is the correct format to specify storage tiers in the GUI or how to manually bypass the GUI and configure this. Thank you Daniel
... View more
Labels: