Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to handle more than nifi.queue.swap.threshold setting

Highlighted

How to handle more than nifi.queue.swap.threshold setting

New Contributor

hi all.

process is ExecuteSQL -> SplitJson -> Queue -> ExecuteGroovyScript ->

nifi.properties file -> nifi.queue.swap.threshold = 20000(default)


If the flowfile is 43508, only the nifi.queue.swap.threshold value is processed, and more is accumulated in the queue and not processed


107782-nifiqueueswapthreshold.png


How to handle more than nifi.queue.swap.threshold setting


If the settings do not resolve..

nifi-app.log written that swap out was successful and two swap files were created, but the swap file was not found when swap in.

2019-04-05 16:31:59,534 WARN [Timer-Driven Process Thread-6] o.a.n.controller.FileSystemSwapManager Cannot swap in FlowFiles from location D:\download\NIFI-1~1.0-B\NIFI-1~1.0\.\flowfile_repository\swap\1554449490032-01691059-3464-1fc9-0a1a-9def8b12c89e-15293fe8-c2ea-4f9d-a4fb-fd992716903d.swap because the FlowFile Repository does not know about this Swap Location. This file should be manually removed. This typically occurs when a Swap File is written but the FlowFile Repository is not updated yet to reflect this. This is generally not a cause for concern, but may be indicative of a failure to update the FlowFile Repository.


But I confirmed that the swap file is in the path

107752-swap.png


Changing nifi.queue.swap.threshold = 30,000 will process up to 30,000 and stop equally.


Thank you for your help.

1 REPLY 1

Re: How to handle more than nifi.queue.swap.threshold setting

New Contributor

I have exactly the same issue... I can increase the nifi.queue.swap.threshold but why does this happen in the first place ? Isn't the swap working at all ?

Don't have an account?
Coming from Hortonworks? Activate your account here