I have just started working on a proof of concept using apache nifi and minifi. My use case involves sending data from and to minifi as well. I managed to create the config as shown in the community to get the data from minifi to nifi however how do i do the reverse? also how do i update/regenerate the minif config with multiple flows defined should I use same minifi process group that was used to extract xml template by adding additional flows.
For eg consider this:
I create new process group and within it I add tailfile->extractext->remoteprocessgroup to nifi
and on nifi I recieve it and do updateattribute->putfile.
I generate the xml for minifi convert it for minifi yaml file and push it and test the workflow. it works fine as expected.
Now I want to do additional step in nifi do a sort of enrichment (i.e after updateattribute on more success relationship to output port) and send it back to minifi. And on minifi I need to regenerate xml this time for addtional steps for recieveing the data and processing them. Please let me know if this possible
Hi @Parthiban Gunasekaran, for sure Site-to-Site communication allows both ways communication.
At the moment you can have MiNiFi sending data to NiFi using a RPG (in MiNiFi) pointing to an input port in NiFi. and, with MiNiFi 0.2, you can add pull data from NiFi using a RPG (in MiNiFi) pointing to an output port in NiFi.
Hope this helps.
I am planning to implement this use case pushing tail output from NiFi (HDF) to MiniFi to HDFS (i.e. HDP). I attempted as per your suggestions, however there are errors and looks like the input port in MiniFi would require. Below is the error & attached the error + ports info
I have 2 clusters, one for NiFi (HDF 3.2) & another HDP 2.6.5
"'Input Ports' in section 'top level' because must be empty in root group as external input/output ports are currently unsupported"
Am I missing something? Kindly guide.
Anybody out there who has implemented this use case?
Pushing data from NiFi -> MiNiFi - please share the flow snapshot and some description would be great help.