Causedby: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory This class is a Phoenix class, when you install Phoenix it sets this class name somewhere in hbase-site.xml and then when NiFi creates the HBase client, the HBase client is reading hbase-site.xml and trying to instantiate this class which doesn't exist in the HBase client. In Apache NiFi 1.1.x there is a property on the HBase client service for specifying the location of the phoenix-client.jar. Setting that should solve this. ... View more

First, its important to understand how consuming from Kafka works in NiFi. The ConsumeKafka section of this post was my attempt to explain it: http://bryanbende.com/development/2016/09/15/apache-nifi-and-apache-kafka The Kafka client automatically assigns partitions to consumers, and 1 partition can only be consumed by 1 consumer. The # of consumers = The # of NiFi nodes X the # of concurrent tasks on ConsumeKafka So if you have a single NiFi node with 1 concurrent task, then the consumer group has 1 consumer. The error means that a consumer pulled some data and went to commit offsets to Kafka, but Kafka hasn't heard from the consumer in longer than the configured session timeout, so Kafka is saying sorry you can't do that because the partition was already reassigned to another consumer, or is possible unassigned if there are no consumers alive. The most common case of this happening in NiFi was when back-pressure occurred between ConsumeKafka and the next processor... back-pressure causes the preceeding processor to not execute until the back-pressure threshold is reduced. So if ConsumeKafka wasn't allowed to run for a couple of minutes, then the back-pressure is cleared and it runs again, you then get this error. The was addressed in a ticket for the upcoming Apache NiFi 1.2.0 release: https://issues.apache.org/jira/browse/NIFI-3189 ... View more

Have you tried installing the hadoop client on the same machine as NiFi and then sending a similar file to HDFS from the command line? In most cases the issue here is the network or disk. ... View more

The documentation of the user defined properties for QueryDatabaseTable says: Initial Max Value Attribute Expression Language Specifies an initial max value for max value columns. Properties should be added in the format `initial.maxvalue.{max_value_column}`. So these are meant to be used to specify the initial value of a max value column. To add attributes to the flow files for down stream processing you can use an UpdateAttribute processor right after QueryDatabaseTable. ... View more

@Gu Gur Since your processor has INPUT_FORBIDDEN, you can never have flow files coming in to your processor, so you should remove the beginning of onTrigger where you have: FlowFile flowFile = aSession.get(); if(flowFile ==null){ return; } That will always return and exit your onTrigger because there are no flow files. Instead you will want to create a new flow file like: FlowFile flowFile = session.create(); ... View more