It seems the same question discussed at   https://community.cloudera.com/t5/Support-Questions/Passing-list-of-directories-to-ListHdfs-Processor/td-p/364798 To address the limited number of tables is to limit the number of files in HDFS terms, Listing strategy can only be controlled by File Filter and file-filter-mode on what can be listed. The Listting processing has a two-step -->What to list " controlled by the filter " -->From Whereto list  "where in your case looks like sub directory under root has a widespread and number of nested sub dirs are huge where the processor is spending time on recursive searching.    If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Chandan      ... View more

@jacektrocinski Yes, Sir, you are right,  I put more focus on read attribute section of the document which has none but missed seeing Additional details.  Kudos!!   ... View more

Hi @ckumar  Yes I have verified. The port is not being used. However nifi throws the error.  ... View more

You can refer to this article and tune the logback.xml file to delete and rollover based log files based on you space constraints https://community.cloudera.com/t5/Community-Articles/Understanding-how-the-logback-xml-configuration-in-NiFi/ta-p/248662   If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Chandan ... View more

Thank you kinds sir! My problem was solved!! ... View more

Disappointing, but what I was starting to suspect after going through CM's console.  Is there general guidance on including those types of files to the NiFi nodes? ... View more

The issue was not related to the size of the queue, but instead to the fact that when to destination that we are sending those logs to had connectivity issues (which was causing quite a lot of timeout errors), NiFi was using a lot of a significant number of threads to retry sending the logs to the InvokeHTTP destination. This high usage of threads was causing ListenSyslog to not function properly - it was dropping the logs that it was supposed to receive. ... View more

@srimanasay, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  ... View more

Correct, it is currently mounted on every node. I would have thought Nifi would dropped the file state once log files are deleted or archived but does not look like it. I could see 15K file states on the  tailing processor.  ... View more

Hi @SAMSAL: I was trying to implement your solution but I didn't find a way to define a SYS_REFCURSOR in Pre-Query statements. Can you share any working simple example using Oracle DBMS? ... View more