@Matt Clarke The second suggestion works as well. I kepp the third one for a next usage. Thanks for all Matt TV. ... View more

@Matt Clarke Thanks for these suggestions. I'm going to try number 2. And could you give me an example of properties for the number 3 and detectduplicate processor ? Thanks, TV ... View more

@Wynner Hi, Memory settings already to 8Go for both. This morning I tried to set Number min and max of Entries and it works !!! Thanks for your help Wynner. Regards Parameters set (so the merge create about 1000 files) Minimum Number of Entries 1 Maximum Number of Entries 2500 Minimum Group Size 0 B Maximum Group Size No value set Max Bin Age 5 min Maximum number of Bins 100 Delimiter Strategy Text ... View more

@Wynner Thanks for your feddback, Here are the properties (I've also tried with 200MB before but it doesn't work). Merge Strategy Bin-Packing Algorithm Merge Format Binary Concatenation Attribute Strategy Keep Only Common Attributes Correlation Attribute Name No value set Minimum Number of Entries 1 Maximum Number of Entries No value set Minimum Group Size 20 MB Maximum Group Size 20 MB Max Bin Age 5 min Maximum number of Bins 100 Delimiter Strategy Text Header No value set Footer No value set Demarcator Compression Level 1 Keep Path false After an hour Nifi fails with a outOfmemory 2017-05-22 12:35:44,770 WARN [NiFi Web Server-22-acceptor-0@2c439296-ServerConnector@ccf1486{HTTP/1.1,[http/1.1]}{0.0.0.0:28080}] o.eclipse.jetty.server.AbstractConnector java.lang.OutOfMemoryError: Java heap space 2017-05-22 12:35:44,784 WARN [NiFi Web Server-21] org.eclipse.jetty.servlet.ServletHandler Error for /nifi-api/flow/controller/bulletins java.lang.OutOfMemoryError: Java heap space at java.lang.StringBuilder.toString(StringBuilder.java:407) ~[na:1.8.0_66] at java.net.Inet4Address.numericToTextFormat(Inet4Address.java:373) ~[na:1.8.0_66] at java.net.Inet4Address.getHostAddress(Inet4Address.java:328) ~[na:1.8.0_66] at org.eclipse.jetty.server.Request.getRemoteAddr(Request.java:1193) ~[na:na] at javax.servlet.ServletRequestWrapper.getRemoteAddr(ServletRequestWrapper.java:275) ~[javax.servlet-api-3.1.0.jar:3.1.0] at org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:62) ~[classes/:na] at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) ~[na:na] at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:316) ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE] ... View more

Exactlty The queue after GetMongo has 2,500,000,evt (1,3GB). The following processor (MergeContent) can not empty this queue. And I don't understand why ? ... View more

I've set it to 100 or 1000 or 2000. After about 10 minutes, the processor reads with success all the collection but in one shot and whatever the value of the propoerty. Is it normal ? ... View more

Hi @Pierre Villard Thanks for your answer. ... View more

Hi @jwit The idea is to match on multiple files. Instead of create several processors with one file to tail for each one, I'd like to create only one processor with a "pattern" to tail which adresses several files. Obviously, we except for each flow ingested the right properties "path, filename ... " of the original file tailed in this flow. So the properties values may change with each flow. Let me know if it's clear to you. Thanks ... View more