create hdfs folder having ':' name,create hdfs folder having ':' in the name

[#1]

Hi,

I was able to create an hdfs folder having space in the name, ex : hdfs dfs -mkdir /tmp/'2017-09-28 12'

But, I want to create a folder having ':' in the name, ex, this does'nt work : hdfs dfs -mkdir /tmp/'2017-09-28 12:02:25'

I need this to create partitions on external hive table using a timestamp.

Many thanks

,

Hi,

I was able to create a folder having space in the name, ex : hdfs dfs -mkdir /tmp/'2017-09-28 12'

But when I introduce ':' in the name, this doesn't work : hdfs dfs -mkdir /tmp/'2017-09-28 12:00:00'

I need this to create partitions on external tables using timestamps.

Many thanks.

[#2]

Yacine Belhoul,

As per Apache Hadoop model we cannot create directories if the filename having / or : characters in it.

HDFS Path elements MUST NOT contain the characters {'/', ':'}

The only way is to replace (:) with %3A

hadoop fs -mkdir /2017-09-28\ \12%3A00%3A09.0

If you do dynamic partition by time stamp field also hive stored these colon(:) replace with %3A hadoop directories.

My dynamic partition time stamp columns are:-

2011-07-07 02:04:51.0

2011-07-07 02:04:52.0

2013-01-30 08:27:16.0

once i'm done with creating dynamic partitions for the table, if i list out the directories they are replace with %3A in place of (:)

Hadoop directories for dynamic partitions:

/apps/hive/warehouse/test_fac/dat=2011-07-07 02%3A04%3A51.0  
/apps/hive/warehouse/test_fac/dat=2011-07-07 02%3A04%3A52.0
/apps/hive/warehouse/test_fac/dat=2013-01-30 08%3A27%3A16.0

Show partitions for dynamic partitioned table:-

if you list the partitions that are in table hive shows those partitions with %3A as a replacement for colon(:)

show partitions test_fac;
+--------------------------------+--+
|           partition            |
+--------------------------------+--+
| dat=2011-07-07 02%3A04%3A51.0  |
| dat=2011-07-07 02%3A04%3A52.0  |
| dat=2013-01-30 08%3A27%3A16.0  |
+--------------------------------+--+

I tried to add paritition to the table

alter table test_fac add partition(dat='2017-09-29 90:00:00');

still it replace colon(:) with %3A.

show partitions test_fac;
+--------------------------------+--+
|           partition            |
+--------------------------------+--+
| dat=2017-09-29 90%3A00%3A00    |
+--------------------------------+--+

but in local file system we can create directories with colon(:) characters in them

Example:-

[~]$ mkdir 2017-09-28\ 12\:00\:09
[~]$ ls
[~]$ 2017-09-28 12:00:09