@Alvaro Dominguez The primary node could change at anytime. You could use postHTTP and listenHTTP processor to route FlowFiles from multiple nodes to a single node. My concern would be heap usage to merge (zip) 160K FlowFiles on a single NiFi node. The FlowFile metadata for all those FlowFiles being zipped would be help in heap memory until the zip is complete. Any objection to having a zip of zips? In other words you could still create 4 unique zip files (1 per node each with unique filename), then send these zipped files to one node to be zipped once more in to a new zip with the single name you want written into HDFS. Thanks, Matt ... View more

@Alvaro Dominguez The only way to create one single zip file is to have one node perform the zipping of all the files. This sounds less then ideal. How large are each of these individual zip files and how many FlowFiles on average go in to each zip file? ... View more

@Alvaro Dominguez Multiple nodes can write to the same path in HDFS, but not the same file at the same time. The lease error you saw above is moist likely the result of one node completing writing .example_1202.zip and then renaming it example_1202.zip. In between that time, a different node saw and then tried to start appending to .example_1202.zip, but it was moved/renamed before that could happen. It essentiall becomes a race condition since nodes do not communicate thsi kind of information with one another. You could write 4 zip files to HDFS every minute. You could just name each filename uniquely based on NiFi hostname writing file. Thanks, Matt ... View more

@Alvaro Dominguez Every node in a NiFi cluster runs its own copy of the cluster flow, has its own repositories, and works on its own set of FlowFiles. Nodes in a NiFi cluster are unaware of any FlowFiles being processed by other nodes in the cluster. What you are seeing is normal expected behavior of your dataflow. Thanks, Matt ... View more

@Oleksandr Solomko Any other ERROR or WARN log messages? --- Is this a standalone NiFi installation or a multi-node NiFi cluster? - If cluster, are all these FlowFiles queued on just one node? --- Is this Nifi secured (HTTPS or HTTP)? I can't reproduce locally. Thanks, Matt ... View more

@Oleksandr Solomko There must be something else going on in your system. Are you seeing any WARN or ERROR log messages in your nifi logs? Did you run out of disk space at any time? Are you seeing any Out Of Memory (OOM) errors in your nifi logs? Thanks, Matt ... View more

@Simran Kaur I had a feeling your issue was related to a missing config. Glad to hear you got it working. If this answer addressed your original question, please mark it as accepted. As far as your other question goes, I see you already started a new question (https://community.hortonworks.com/questions/105720/nifi-stream-using-listenhttp-processor-creates-too.html). That is the correct approach in this forum, we want to avoid asking unrelated questions in the same post. I will have a look at that post as well. Thank you, Matt ... View more

@Oleksandr Solomko What version of NiFi are you running? Are you seeing any Out Of Memory errors in your nifi logs? This could be causing issues with emptying the queue. The fastest way to clear this specific queue now might be to stop both the SplitAVRO and PublishKafka processors. Add an UpdateAttribute processor to your graph with the success relationship set to auto-terminate. Select the queued connection and drag the blue dot (near the arrow end of the connection) to the update Attribute processor. Start the updateAttribute processor and it will start auto-terminating FlowFiles form this connection in batches. Thanks, Matt ... View more