Hi @jgw.gooijer Is there any relationship between 'the flow of a month' and the file names of FlowFiles those are listed by FTP and waiting at the Wait processor? If you can derive year and month from a waiting listed file name, by using Attribute Expression Language, then you can configure Wait/Notify processors to use 'year-month' value as signal identifier. This way, you don't have to close identifiers. Keep them open after 'the flow of a month' finished, so that any number of FlowFiles having the same 'year-month' can go through Wait processor. ... View more

I don't have much experience with Apache Drill, but looking at the source code, it looks like it fails at instantiating the class specified with 'hadoop.security.group.mapping' parameter. The parameter is usually defined at a Hadoop configuration file, 'core-default.xml'. Are you aware of any configuration file specifying 'hadoop.security.group.mapping' parameter potentially being null? ... View more

I don't have a Drill environment to test with, but downloaded Drill JDBC driver and tested if I can load its Driver class with NiFi DBCPConnectionPool. Since drill driver needs other dependency jars, I needed to use the jar which contains all dependencies. Please try drill-jdbc-all-1.13.0.jar instead. apache-drill-1.13.0/jars/jdbc-driver/drill-jdbc-all-1.13.0.jar ... View more

@Tommy The error message has been changed slightly, it seems the jar file is used now but failed with different reason. Would you please share the detail of the error? I think nifi-app.log has a stack-trace for the exception, which may be different than the one you posted before. ... View more

@Tommy Please make sure that the jar file path is correct. If you are using a multi-node NiFi cluster, each node has to have the jar file at the specified path. ... View more

Hi @Tommy In the screenshot, I see you set /var/lib/dlfs/lib directory at 'Database Driver Location(s)'. How about setting the jar file path instead? ... View more

Hi @Sam Rubin I took a look on the shell output and found -Xms and -Xmx are being configured with quite small amount: ++ /bin/java -cp '/apps/nifi/nifi-1.5.0/conf:/apps/nifi/nifi-1.5.0/lib/bootstrap/*' -Xms12m -Xmx24m -Dorg.apache.nifi.bootstrap.config.log.dir=/apps/nifi/nifi-1.5.0/logs -Dorg.apache.nifi.bootstrap.config .pid.dir=/apps/nifi/nifi-1.5.0/run -Dorg.apache.nifi.bootstrap.config.file=/apps/nifi/nifi-1.5.0/conf/bootstrap.conf org.apache.nifi.bootstrap.RunNiFi start It may not be the direct cause of having start/shutdown loop, but NiFi wouldn't run with 24m heap size. Is this configuration intentional?? Please check bootstrap.conf and increase -Xms and -Xmx to 512m at least. ... View more

Thanks @Sam Rubin for the log. I couldn't find anything unusual in the logs. By looking at NiFi source code, the "Failed to send shutdown command to port ... Will kill ..." message is only logged by calling RunNiFi.stop() method. And the method is only called from RunNiFi.main() which is called by nifi.sh. Would you run the nifi.sh with debug option? Such as: sh -bx ./bin/nifi.sh start Please share std output of above command. Then we can look how the nifi.sh works in detail. Just in case, did you modify nifi.sh? ... View more

Hi @Sam Rubin, I see the WARN message regularly when NiFi restarts if target remote NiFi process group is not running, but haven't encountered start/shutdown loop because of it. The [RemoteProcessGroup47a43fe6-2c9b-39aa-4fae-5851f048d58bThread-1] that logged the WARN message is a background thread and periodically (per 60 secs) checks target RPG status, and does not affect the main thread. Is it possible for you to share entire nifi-app.log and nifi-bootstrap.log? I suspect there is other issues preventing NiFi to startup correctly. ... View more