Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Fork and Join with Nifi

Highlighted

Fork and Join with Nifi

Explorer

With Oozie we can use fork and join to launch parallel actions. All launched parallel actions will join a final Action. The Final action will begin to work when all parallel actions are finished successfully.

Is there an elegant solution for having the same feature for processors with nifi ?

1 REPLY 1
Highlighted

Re: Fork and Join with Nifi

NiFi is not really a parallel job execution framework, it is a data flow tool that moves data from one place to another and can perform operations on that data along the way. There is a way to parallelize some operations such as the List+Fetch model, where a processor on the primary node performs a listing, for example list the contents of an HDFS directory, then distribute those results to all the nodes in the cluster to perform the fetch in parallel (see this post https://community.hortonworks.com/articles/16120/how-do-i-distribute-data-across-a-nifi-cluster.html..., but there is not a concept of final action.

Don't have an account?
Coming from Hortonworks? Activate your account here