NiFi FetchFTP retention on FTP 550 The system cannot find the file specified.

[#1]

Hello,

we have setup retention on our source FTP and nifi-app.log are full of events

2021-01-27 10:00:09,281 ERROR [Timer-Driven Process Thread-86] o.a.nifi.processors.standard.FetchFTP FetchFTP[id=11ee37bc-fda2-1627-b55b-bbbc5b85c7be] Failed to fetch content for StandardFlowFileRecord[uuid=4da465bb-a4cf-4a79-9020-0e2775517caa,claim=,offset=0,name=lte14-pgwcdr-b20210126221809-1269.dat.tmp,size=0] from filename /DWHSK_Extracts/MEDIACIA/CDR/GPRS/TMP/lte14-pgwcdr-b20210126221809-1269.dat.tmp on remote host NTDWHFS401:21 due to java.io.IOException: 550 The system cannot find the file specified.
; routing to comms.failure: java.io.IOException: 550 The system cannot find the file specified.

java.io.IOException: 550 The system cannot find the file specified.

Is there any way to turn off this ERROR rows or any alternative like turn off logging FetchFTP processor/custom log file for this processor?

Best regards

Kamil

[#2]

By looking at request ,I assume flow design would be ListFTP--->FetchFTP , So after listing of file by  ListFTP -->flow files passed to FetchFTP to fetch from FTP, but file is deleted by retention policy before FetchFTP try to fetch is expected result " resulting processor throwing an error "The system cannot find the file specified"

If error is expected and you just want to suppress those errors which is genuine one , I do not see this is a good idea so when if other issue encounter by FetchFTP , user won't be able to notified.

Also I do not see this is possible. 

[#3]

Hello,

thank for answer. Is there any way to route FetchFTP log to separate log file? 

There are logs full of these errors now. Is hard to find some meaningfull in.

Regards

Kamil

[#4]

@Kamil_Valko 

All NiFi logging is handled by the logback.xml file.

By default you will find that NiFi's logback.xml contains three unique file appenders.
1. APP_FILE
2. USER_FILE
3. BOOTSTRAP_FILE.

To log anything to a different file, the first thing to do is create another appender.  I recommend copying the APP_FILE appender.

<appender name="FTP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
            <file>/var/log/nifi/nifi-ftp.log</file>
            <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
            <fileNamePattern>/var/log/nifi/nifi-ftp_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern>
            <maxFileSize>100MB</maxFileSize>
            <maxHistory>5</maxHistory>
            <totalSizeCap>1GB</totalSizeCap>
            </rollingPolicy>
            <immediateFlush>true</immediateFlush>
            <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
            </encoder>
            </appender>

Then you need to add a new logger for the FetchFTP processor class.  You cannot control logging for specific instance of a processor.  This means all logging from all instantiations of the listFTP processor on the canvas will get handled by this new logger.

            <logger name="org.apache.nifi.processors.standard.FetchFTP" level="ERROR" additivity="false">
            <appender-ref ref="FTP_FILE"/>
            </logger>

You can turn off logging for a specific processor class, but I agree with @ckumar that this is not an ideal path.  But for completeness of this response, setting level to "OFF" on a logger for a specific class will stop all logging from that class. (Remember that this affects logging from all occurrences of this processor used by this NiFi)

Hope this helps,
Matt