Support Questions

Find answers, ask questions, and share your expertise

How to get the files from HDFS ingested on a particular day or time?

I am ingesting messages from MQ to HDFS (to a staging area). I want to copy these files to local. The issues is the ingestion process runs 24x7. I only want files ingested on a particular date (or between a particular time on that day) i.e. 23rd April, 2019 etc. Is there a way to copyToLocal/get the files/messages from HDFS ingested/created on a particular date?

Doing it manually is a pain.

1 REPLY 1

Explorer

Hi Chandan

I also have a similar requirement now and understood that find command in HDFS has limited opitons.

Did you find any way to solve the above problem? Thanks. Muthu

https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/FileSystemShell.html#find