Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to send data from Nifi to Minifi

Highlighted

How to send data from Nifi to Minifi

New Contributor

I have just started working on a proof of concept using apache nifi and minifi. My use case involves sending data from and to minifi as well. I managed to create the config as shown in the community to get the data from minifi to nifi however how do i do the reverse? also how do i update/regenerate the minif config with multiple flows defined should I use same minifi process group that was used to extract xml template by adding additional flows.

For eg consider this:

I create new process group and within it I add tailfile->extractext->remoteprocessgroup to nifi

and on nifi I recieve it and do updateattribute->putfile.

I generate the xml for minifi convert it for minifi yaml file and push it and test the workflow. it works fine as expected.

Now I want to do additional step in nifi do a sort of enrichment (i.e after updateattribute on more success relationship to output port) and send it back to minifi. And on minifi I need to regenerate xml this time for addtional steps for recieveing the data and processing them. Please let me know if this possible

4 REPLIES 4
Highlighted

Re: How to send data from Nifi to Minifi

Hi @Parthiban Gunasekaran, for sure Site-to-Site communication allows both ways communication.

At the moment you can have MiNiFi sending data to NiFi using a RPG (in MiNiFi) pointing to an input port in NiFi. and, with MiNiFi 0.2, you can add pull data from NiFi using a RPG (in MiNiFi) pointing to an output port in NiFi.

Hope this helps.

Highlighted

Re: How to send data from Nifi to Minifi

Explorer

@Pierre Villard

I am planning to implement this use case pushing tail output from NiFi (HDF) to MiniFi to HDFS (i.e. HDP). I attempted as per your suggestions, however there are errors and looks like the input port in MiniFi would require. Below is the error & attached the error + ports info

I have 2 clusters, one for NiFi (HDF 3.2) & another HDP 2.6.5

"'Input Ports' in section 'top level' because must be empty in root group as external input/output ports are currently unsupported"

93356-error.png

Am I missing something? Kindly guide.

nifi-1.pngnifi-2.pngnifi-3.png

Thanks,

Bharat

Highlighted

Re: How to send data from Nifi to Minifi

New Contributor

Hi Parthiban,

I need to assistance on sending the data from MiniFy to Nifi . I am new to this tool. Kindly assists me the flow details from MiniFi to Nifi.

I followed below document but its not working for me after did the proper setups.

https://nifi.apache.org/minifi/download.html

Kindly advise.

Highlighted

Re: How to send data from Nifi to Minifi

New Contributor
Don't have an account?
Coming from Hortonworks? Activate your account here