How exactly does the PutKafka processor User-Defined partition work in NiFi?

[#1]

I am trying to see if I could use the User-Defined partition in NiFi to route based on a field in an incoming flowfile that is getting sent to the PutKafka processor. The field we would use is an attribute in the flowfile. Would this be possible using the User-Defined partition in the PutKafka processor? I am having doubts of how exactly does the User-Defined partition work in NiFi? Would I have to set the Partition and Kafka key in the configuration in order to use it?

[#2 · Accepted solution]

I believe you can do what you described... You would select "User-Defined" as the partition strategy, which means it will send to the partition provided in the "Partition" property. Since "Partition" supports expression language, this could be a reference to an attribute of an incoming flow file such as ${partition}.

If I am understanding your scenario correctly, lets say you flow files with a "type" attribute that can be either "A" or "B", and you want all "A" messages to go to partition 1 and all B messages to partition 2. You could use a RouteOnAttribute processor to route A message to one relationship, and B messages to the other, and then for each of them have an UpdateAttribute processor that adds a new attribute like partition = 1 or partition = 2, so that when it gets to PutKafka you can reference ${partition}.

[#3]

Thanks @Bryan Bende , your answer was really helpful!