Member since
08-27-2019
6
Posts
0
Kudos Received
0
Solutions
08-09-2020
09:27 PM
I have flow files with different dimensions. But they have a common id column. I want to use that to join the flowfiles and pick specific columns. How can I use mergeContent in this case?
... View more
08-29-2019
07:05 AM
1 Kudo
Hello @Teej The short answer is that FetchX (FetchFTP for example) is Nifi cluster friendly, while GetX processors are not. There is a common pattern ("List-Fetch") of using a single node to ListX then pass that List to all nodes in the cluster to do parallelized FetchX - the Fetch will be aware that there are multiple nodes and only Fetch each file once. If you have a NiFi cluster and you are using the GetSFTP processor, you would have to configure that processor to run on the primary node only so the other nodes in the cluster wouldn't try to pull the same files. You can read more about it here.
... View more
08-29-2019
03:11 AM
A quick search suggests that libhdfs can do it, but I have not tried it myself.
... View more