I have the following Nifi Flow:
QueryDatabaseTable -> AvroToJSON -> SplitJSON -> ExtractText (write value to attribute) -> FetchDistributedMapCache (check if the attribute is available in cache, which is set in another flow) -> UpdateRecord (Rename some fields) -> MergeContent -> PutDatabaseRecord
And sometimes this error occurs:
The FlowFile is not routed to failure, but kept in the queue and the Processor is now stuck in an endless loop trying to process the faulty FlowFile.
When i skip the MergeContent/MergeRecord Processor and just write to the database for each record this error does not occurr, but writing for each object is very slow.
Why does this happen? How can I debug this?
Hi, I hope you have find out what was going on... by the way, I have same problem with flowfiles not going to failure in putHiveQl and putHDFS, they just stuck and dont go to failure.
Do you/someone have any ideia why is this happening?
Tks a lot!
Is there anything in the logs before/after the "already marked for transfer" entry? Trying to figure out how a flow file can get transferred and then something goes wrong (where we'd try to also send it to failure)