Hello Unfortunately this means the process NiFi was told to execute has not returned. In such a case there is an outstanding thread and we intentionally prevent additional instances from being started until this one is dealt with. From the stacktrace you provided we see "Timer-Driven Process Thread-9" Id=69 RUNNABLE (in native code) at java.io.FileInputStream.readBytes(Native Method) at java.io.FileInputStream.read(FileInputStream.java:272) at java.io.BufferedInputStream.read1(BufferedInputStream.java:273) at java.io.BufferedInputStream.read(BufferedInputStream.java:334) - waiting on java.lang.UNIXProcess$ProcessPipeInputStream@e6f0a2f This tells us we're sitting and waiting for that command to do something (finish, respond with data). It appears to be in a hung state. You'll need to restart NiFi and try to assess why that command isn't doing anything. What command are you trying to run? It might be that for your case the 'ExecuteProcess' processor is a better fit. Thanks Joe ... View more

Hello, Keep in mind each FlowFile is comprised of attributes and content. RouteOnAttribute is built specifically to look at FlowFile attributes. RouteOnContent looks at FlowFile content. In the case of JSON you can, for example, use EvaluateJSONPath to extract fields from the JSON to become FlowFile attributes. You can use SplitJSON to split the JSON bundles into invidual flowfiles then extract values and then route on attributes. The various processors offer you a variety of ways to route, transform, and deliver the data. Thanks Joe ... View more

This is due to this issue https://issues.apache.org/jira/browse/NIFI-990 which fixed the fact that the failure relationship was mistakenly not provided before. When importing templates made against the previous version they will not have that relationship checked since it wasn't there. So it will show as invalid until you check it as auto-terminate or use it. You may wish to recreate the template. ... View more

The HDF release does support interacting with Kerberized Kafka instances as found within the HDP stack. This is because HDP added support for Kerberized Kafka in advance of the community supporting it (Kafka 0.8.x). In Apache Kafka world now (0.9x) there is Kerberos support. So, - Apache NiFi supports non Kerberized Kafka clusters today. - HDF releases of NiFi have patched support for Kerberized Kafka clusters in HDP - Upcoming Apache NiFi releases will add support for the 0.9x Apache Kafka kerberos model Thanks Joe ... View more

I haven't tested this myself but NAS/SAN arrangements have worked quite well in the past. Needs testing to understand latencies/tradeoffs but frankly I suspect it will work just fine. ... View more

In the Get/PutKafka processors you should be able to add a dynamic property called 'fetch.message.max.bytes' and set the value you need. The processor should allow you to add dynamic properties which map to Kafka consumer properties and it will pass them to the consumer/producer config as needed. ... View more

Vance, We completely agree with you. NiFi already supports some powerful security and multi-role authorization capabilities. But as you mention we should support multiple different groups with different levels of access to various parts of the flow. That is an important roadmap item and work is underway. You can see a bit about the nifi community thinking on this wiki page https://cwiki.apache.org/confluence/display/NIFI/Multi-Tentant+Dataflow and there are related threads such as https://cwiki.apache.org/confluence/display/NIFI/Redesign+User+Interface and https://cwiki.apache.org/confluence/display/NIFI/Support+Authorizer+API If you need help setting up secure NiFi you can read more here https://community.hortonworks.com/articles/886/securing-nifi-step-by-step.html and in the administration guide https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#security-configuration Thanks Joe ... View more