How to send data from Nifi to Minifi

[#1]

I have just started working on a proof of concept using apache nifi and minifi. My use case involves sending data from and to minifi as well. I managed to create the config as shown in the community to get the data from minifi to nifi however how do i do the reverse? also how do i update/regenerate the minif config with multiple flows defined should I use same minifi process group that was used to extract xml template by adding additional flows.

For eg consider this:

I create new process group and within it I add tailfile->extractext->remoteprocessgroup to nifi

and on nifi I recieve it and do updateattribute->putfile.

I generate the xml for minifi convert it for minifi yaml file and push it and test the workflow. it works fine as expected.

Now I want to do additional step in nifi do a sort of enrichment (i.e after updateattribute on more success relationship to output port) and send it back to minifi. And on minifi I need to regenerate xml this time for addtional steps for recieveing the data and processing them. Please let me know if this possible

[#2]

Hi @Parthiban Gunasekaran, for sure Site-to-Site communication allows both ways communication.

At the moment you can have MiNiFi sending data to NiFi using a RPG (in MiNiFi) pointing to an input port in NiFi. and, with MiNiFi 0.2, you can add pull data from NiFi using a RPG (in MiNiFi) pointing to an output port in NiFi.

Hope this helps.

[#3]

@Pierre Villard

I am planning to implement this use case pushing tail output from NiFi (HDF) to MiniFi to HDFS (i.e. HDP). I attempted as per your suggestions, however there are errors and looks like the input port in MiniFi would require. Below is the error & attached the error + ports info

I have 2 clusters, one for NiFi (HDF 3.2) & another HDP 2.6.5

"'Input Ports' in section 'top level' because must be empty in root group as external input/output ports are currently unsupported"

Am I missing something? Kindly guide.

nifi-1.pngnifi-2.pngnifi-3.png

Thanks,

Bharat

[#4]

Hi Parthiban,

I need to assistance on sending the data from MiniFy to Nifi . I am new to this tool. Kindly assists me the flow details from MiniFi to Nifi.

I followed below document but its not working for me after did the proper setups.

https://nifi.apache.org/minifi/download.html

Kindly advise.

[#5]

https://nifi.apache.org/minifi/getting-started.html

[#6]

Anybody out there who has implemented this use case?

Pushing data from NiFi -> MiNiFi - please share the flow snapshot and some description would be great help.

Thanks,

-Maqbool.