question Nifi : putHDFS losing Data in Archives of Support Questions (Read Only)

Nifi : putHDFS losing Data

maykiwogno — Tue, 08 Nov 2016 02:43:26 GMT

Hi All,

When putHdfs Process with Data what happen if the Nifi node crash or if fail to write to HDFS ?

It is possible to rerun a specific flowfile ?

Regards

Re: Nifi : putHDFS losing Data

bbende — Tue, 08 Nov 2016 02:52:10 GMT

If it failed to write the data to HDFS it would transfer the flow file to the failed relationship, which you route back to itself to keep re-trying, or route to whatever you want to do to handle the error.

If the NiFi node crashed while it was in process of writing the data, but hadn't committed the session in the processor, then when the node restarts that flow file will still be in the queue before the processor and will be tried again.

There should not be data loss.

Re: Nifi : putHDFS losing Data

maykiwogno — Tue, 08 Nov 2016 04:49:37 GMT

Thanks. It is very clear.