https://community.cloudera.com/t5/Archives-of-Support-Questions/Nifi-ConsumeKafka-processor/m-p/172157#M45899 <P>Hi, </P><P>Could someone shed some light on how the above processor initiates the commit? Is there a way to enforce commit only after a set of subsequent processors completes successfully, for e.g. PutHDFS?</P> Fri, 11 Nov 2016 01:25:52 GMT sivaraman_js 2016-11-11T01:25:52Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Nifi-ConsumeKafka-processor/m-p/172157#M45899 <P>Hi, </P><P>Could someone shed some light on how the above processor initiates the commit? Is there a way to enforce commit only after a set of subsequent processors completes successfully, for e.g. PutHDFS?</P> Fri, 11 Nov 2016 01:25:52 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Nifi-ConsumeKafka-processor/m-p/172157#M45899 sivaraman_js 2016-11-11T01:25:52Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Nifi-ConsumeKafka-processor/m-p/172158#M45900 <P>ConsumeKafka commits the offsets to Kafka right after the data has been written to flow file and the session for that flow flow has been committed. This way there is no chance for the data to be lost before committing the offsets to Kafka because the data has already been persisted to NiFi's repositories.</P><P>Currently there is not a concept of having a series of processors treated as one operation. Right now you can think of it as two separate transfers of data, the first being from Kafka to NiFi, the second from NiFi to HDFS.</P> Fri, 11 Nov 2016 01:34:59 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Nifi-ConsumeKafka-processor/m-p/172158#M45900 bbende 2016-11-11T01:34:59Z