Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi Cannot swap in FlowFiles from location

Nifi Cannot swap in FlowFiles from location

New Contributor
107641-nifiqueueswapthreshold.png


107651-swap.png


When 43,508 flow files are sent to SplitJson, they are stacked in the queue and processed within the value set in nifi.queue.swap.threshold(= 20000).

After that, a swap file was created and there are 23,508 stacks in the queue

but it is not running any more and is stopped.


nifi-app.log seems to be in progress because of an error in swap in.

2019-04-05 16:31:59,534 WARN [Timer-Driven Process Thread-6] o.a.n.controller.FileSystemSwapManager Cannot swap in FlowFiles from location D:\download\NIFI-1~1.0-B\NIFI-1~1.0\.\flowfile_repository\swap\1554449490032-01691059-3464-1fc9-0a1a-9def8b12c89e-15293fe8-c2ea-4f9d-a4fb-fd992716903d.swap because the FlowFile Repository does not know about this Swap Location. This file should be manually removed. This typically occurs when a Swap File is written but the FlowFile Repository is not updated yet to reflect this. This is generally not a cause for concern, but may be indicative of a failure to update the FlowFile Repository.


Changing nifi.queue.swap.threshold = 40,000 will process up to 40,000 and stop equally.



Thank you for your help.

4 REPLIES 4

Re: Nifi Cannot swap in FlowFiles from location

New Contributor

We have the same issue, has anyone found any solution to this case?

Re: Nifi Cannot swap in FlowFiles from location

New Contributor

This seems to happen since 1.9, I reverted back to 1.8.0 and it runs again.

Before I did this, I turned off the caching for any repo in the nifi.properties. Had no effect on this issue, only made Nifi extremely slow (as expected)

Highlighted

Re: Nifi Cannot swap in FlowFiles from location

New Contributor

Breakdown down your problem into layers.after splitjson try ingesting into Kafka/HDFS then you understand why its not swapping.


Re: Nifi Cannot swap in FlowFiles from location

New Contributor

After read the content of the link below a made some changes in both nifi.properties e boostrap.conf files. That changes solved the issue, for now.

https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.h...


nifi.properties file:

<nifi.queue.swap.threshold=100000/>


Bootstrap.conf file:

<# JVM memory settings

java.arg.2=-Xms4g

java.arg.3=-Xmx4g/>


coment Garbage collector usage

<# java.arg.13=-XX:+UseG1GC/>