- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to copy files recursive from HDFS to a local folder?
Created 02-25-2019 10:50 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The simple way to copy a folder from HDFS to a local folder is like this:
su hdfs -c 'hadoop fs -copyToLocal /hdp /tmp'
In the example above, we copy the hdp
folder from HDFS to /tmp/local_folder
But we have a more complicated case. Let’s say under /hdp ( hdfs )
, we have subfolders:
/hdp/folder1
/hdp/folder2
/hdp/folder3/folder_a
.
.
.
And in each sub folder we have files.
In that case, how to copy recursively only the files under /hdp
to the local dir /tmp/local_folder
?
Created 02-26-2019 02:12 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @Michael Bronson,
We could suffix "*" (star) to the folder /hdp which copies only the files under /hdp recursively to local_folder.
hadoop fs -copyToLocal /hdp/* /tmp/local_folder
Created 02-26-2019 02:17 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
but in case under /hdp we have also sub folders , then what we can do ?
