I am looking to create a customNIFI processor. It receives flowfiles, but in order to generate the resulting flowfile, it must first reach out to a backend service to get information that must be added to the output flowfile.
I am currently thinking of accessing the backend service in a worker thread in the custom processor. Once the additional information is received, I'd then queue this for an asynchronous creation of an output flowfile (in onTrigger).
Is this a reasonable thing to do in a custom processor? Typically (using the egress pattern - or many of the other common patterns- documented on the NIFI developer page), input flowfiles are immediately (synchronously) transformed into an output flowfile.
However that is not the pattern I have in mind.. Input flowfiles will instead be asynchrnously transformed into output flowfiles (and this might not even be a 1 to 1 mapping from input flowfile to output flowfile). In this regard the processor will have behaviour similar to MergeContent, or MergeRecord (which are indeed asynchronously generating output flowfiles).