https://community.cloudera.com/t5/Support-Questions/ListFile-on-a-huge-directory/m-p/296740#M218391 <P>Hi, I'm trying to do a recursive list file on a massive directory. We're talking close to 100 GB. It seems like ListFile is walking through every single solitary file before it will spit out the first flow file. I was assuming it would spit out files one at a time as it finds them.&nbsp;</P><P>&nbsp;</P><P>It seems as though there's simply too much data. It just kind of sits there and spins. I left it run over the weekend and it never got through everything or spit anything out.&nbsp; I can't tell if its still working or if it's overloaded or what is going on.</P><P>&nbsp;</P><P>How would I go about walking through a directory like this returning one file at a time without duplicates?</P> Wed, 27 May 2020 19:25:44 GMT ischuldt24 2020-05-27T19:25:44Z