Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

how to use path globs to copy files from local to hdfs???

avatar
Expert Contributor

I have some number of files having format like

1)filename+date.filefomat

2)filename.fileformat

now i need to copy only files which have some number before .(dot).

1 ACCEPTED SOLUTION

avatar

You can do something like this:

hdfs dfs -put *filename*[0-9].txt /tmp

For example:

$ touch ${RANDOM}filename-$(date +"%F").txt ${RANDOM}filename.txt
$ ls *filename*.txt
17558filename-2016-01-05.txt  27880filename.txt
$ hdfs dfs -put *filename*[0-9].txt /tmp
$ hdfs dfs -ls /tmp
-rw-r--r--   3 hdfs      hdfs          0 2016-01-05 16:39 /tmp/17558filename-2016-01-05.txt

If that doesn't work, add this to the beginning:

set +f

View solution in original post

1 REPLY 1

avatar

You can do something like this:

hdfs dfs -put *filename*[0-9].txt /tmp

For example:

$ touch ${RANDOM}filename-$(date +"%F").txt ${RANDOM}filename.txt
$ ls *filename*.txt
17558filename-2016-01-05.txt  27880filename.txt
$ hdfs dfs -put *filename*[0-9].txt /tmp
$ hdfs dfs -ls /tmp
-rw-r--r--   3 hdfs      hdfs          0 2016-01-05 16:39 /tmp/17558filename-2016-01-05.txt

If that doesn't work, add this to the beginning:

set +f