When to use "primary node only" depends on whether the operation is something that makes sense to happen on all nodes, or whether its something that only makes sense to happen once. Here are some examples... ListHDFS - this should be primary node only because otherwise you are going to perform the same listing on all nodes ConsumeKafka - this can be run on all nodes because each one will be consuming different data GetFile - this can be run on all nodes because each node will pick up different data from a local directory In your Kafka scenario, instances of a processor equate to what you see on the graph times the # of nodes in the cluster, so if you have a two node cluster with one ConsumeKafka_0_10 on the canvas, then there are two instances of ConsumeKafka_0_10. If you increase concurrent tasks to 3, then there are 3 threads executing each instance on each node, so 6 total. Since you have 6 partitions, each of these 6 threads should consume from a separate partition.
... View more
