I am ingesting messages from MQ to HDFS (to a staging area). I want to copy these files to local. The issues is the ingestion process runs 24x7. I only want files ingested on a particular date (or between a particular time on that day) i.e. 23rd April, 2019 etc. Is there a way to copyToLocal/get the files/messages from HDFS ingested/created on a particular date?
Doing it manually is a pain.
I also have a similar requirement now and understood that find command in HDFS has limited opitons.
Did you find any way to solve the above problem? Thanks. Muthu