Member since
04-05-2016
130
Posts
93
Kudos Received
29
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3740 | 06-05-2018 01:00 AM | |
5098 | 04-10-2018 08:23 AM | |
5607 | 07-18-2017 02:16 AM | |
2885 | 07-11-2017 01:02 PM | |
3305 | 07-10-2017 02:10 AM |
07-05-2018
11:37 PM
Hi @jgw.gooijer Is there any relationship between 'the flow of a month' and the file names of FlowFiles those are listed by FTP and waiting at the Wait processor? If you can derive year and month from a waiting listed file name, by using Attribute Expression Language, then you can configure Wait/Notify processors to use 'year-month' value as signal identifier. This way, you don't have to close identifiers. Keep them open after 'the flow of a month' finished, so that any number of FlowFiles having the same 'year-month' can go through Wait processor.
... View more
06-20-2018
12:01 PM
I don't have much experience with Apache Drill, but looking at the source code, it looks like it fails at instantiating the class specified with 'hadoop.security.group.mapping' parameter. The parameter is usually defined at a Hadoop configuration file, 'core-default.xml'. Are you aware of any configuration file specifying 'hadoop.security.group.mapping' parameter potentially being null?
... View more
06-20-2018
07:56 AM
I don't have a Drill environment to test with, but downloaded Drill JDBC driver and tested if I can load its Driver class with NiFi DBCPConnectionPool. Since drill driver needs other dependency jars, I needed to use the jar which contains all dependencies. Please try drill-jdbc-all-1.13.0.jar instead. apache-drill-1.13.0/jars/jdbc-driver/drill-jdbc-all-1.13.0.jar
... View more
06-20-2018
06:53 AM
@Tommy The error message has been changed slightly, it seems the jar file is used now but failed with different reason. Would you please share the detail of the error? I think nifi-app.log has a stack-trace for the exception, which may be different than the one you posted before.
... View more
06-20-2018
05:29 AM
@Tommy Please make sure that the jar file path is correct. If you are using a multi-node NiFi cluster, each node has to have the jar file at the specified path.
... View more
06-15-2018
01:27 AM
Hi @Tommy In the screenshot, I see you set /var/lib/dlfs/lib directory at 'Database Driver Location(s)'. How about setting the jar file path instead?
... View more
06-13-2018
02:51 AM
Hi @Sam Rubin I took a look on the shell output and found -Xms and -Xmx are being configured with quite small amount: ++ /bin/java -cp '/apps/nifi/nifi-1.5.0/conf:/apps/nifi/nifi-1.5.0/lib/bootstrap/*' -Xms12m -Xmx24m -Dorg.apache.nifi.bootstrap.config.log.dir=/apps/nifi/nifi-1.5.0/logs -Dorg.apache.nifi.bootstrap.config
.pid.dir=/apps/nifi/nifi-1.5.0/run -Dorg.apache.nifi.bootstrap.config.file=/apps/nifi/nifi-1.5.0/conf/bootstrap.conf org.apache.nifi.bootstrap.RunNiFi start It may not be the direct cause of having start/shutdown loop, but NiFi wouldn't run with 24m heap size. Is this configuration intentional?? Please check bootstrap.conf and increase -Xms and -Xmx to 512m at least.
... View more
06-09-2018
07:18 AM
Thanks @Sam Rubin for the log. I couldn't find anything unusual in the logs. By looking at NiFi source code, the "Failed to send shutdown command to port ... Will kill ..." message is only logged by calling RunNiFi.stop() method. And the method is only called from RunNiFi.main() which is called by nifi.sh. Would you run the nifi.sh with debug option? Such as: sh -bx ./bin/nifi.sh start Please share std output of above command. Then we can look how the nifi.sh works in detail. Just in case, did you modify nifi.sh?
... View more
06-08-2018
12:29 AM
Hi @Sam Rubin, I see the WARN message regularly when NiFi restarts if target remote NiFi process group is not running, but haven't encountered start/shutdown loop because of it. The [RemoteProcessGroup47a43fe6-2c9b-39aa-4fae-5851f048d58bThread-1] that logged the WARN message is a background thread and periodically (per 60 secs) checks target RPG status, and does not affect the main thread. Is it possible for you to share entire nifi-app.log and nifi-bootstrap.log? I suspect there is other issues preventing NiFi to startup correctly.
... View more
06-05-2018
01:00 AM
1 Kudo
Hi @Óscar Andreu, thank you for reporting the issue. I was able to reproduce the issue. By looking at the code, there is a bug in how JoltTransformJSON validates the spec when it contains EL. The processor tries to check if the JoltSpec contains any custom transformation class to fail fast, but it fails to get a JoltSpec because EL can not return a valid spec at the validation phase. I submitted an Apache JIRA for this and will try to fix it. https://issues.apache.org/jira/browse/NIFI-5268 Thanks!
... View more