It seems the same question discussed at https://community.cloudera.com/t5/Support-Questions/Passing-list-of-directories-to-ListHdfs-Processo...
To address the limited number of tables is to limit the number of files in HDFS terms, Listing strategy can only be controlled by File Filter and file-filter-mode on what can be listed.
The Listting processing has a two-step
-->What to list " controlled by the filter "
-->From Whereto list "where in your case looks like sub directory under root has a widespread and number of nested sub dirs are huge where the processor is spending time on recursive searching.
If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post.
Thank you,
Chandan