About benjbenj

benjbenj · ‎06-11-2022

@MattWho Thanks Matt, I understand what you're saying. in my case it's the other way around, I have clients (MiNiFi) initiating the connections when they want to send or receive data. This helps me deal with remote firewalls not allowing incoming connections and sporadic availability as they're the ones starting the connection. At the moment, with S2S, The MiNiFi's connect periodically (every 10s) to see I there are any flowfiles waiting for them. I would need to figure out how to periodically generate that action to perform the same flow. Example scenarios: 1. Client (MiNiFi side) is running a tail on a file. A new line comes into the file and needs to be sent over to NiFi for processing. 2. NiFi wants to send a file to the remote MiNiFi running a putfile processor. I have both scenarios working right now with S2S with no issues to a bunch of MiNiFi's, it's just that this requires hacky workarounds (scripts to modify the MiNiFi yml) to make this work and I would love to figure out a cleaner solution.

benjbenj · ‎06-09-2022

@araujo that's an interesting way to go about it. I haven't tried MTLS between InvokeHTTP and ListenHTTP, but i assume that's doable within the SSL context service. At the moment, my MiNiFi clients connect periodically (every 10s) to see if there's anything queued for them from the NiFi side (each have their own output port in this config), i guess i would have to trigger that periodic fetch somehow... You've given me stuff to think about. thanks. 🙂

benjbenj · ‎06-08-2022

@Wynner i worked around this by having a process on the MiNiFi to update the local config.yml before starting. Basically i change out the connection source id for the processor i need. It's a fairly hacky way to make this work, but it's working and i haven't had the time or reason yet to find a cleaner way to get this done. if you find something, please come back here and comment. 🙂

benjbenj · ‎06-01-2022

Anyone coming to look at this now (Jun 2022) - You can simply use UpdateAttribute with ${test:padLeft(10, '0')} to pad a string with zeroes.

benjbenj · ‎08-07-2021

Hi, I want to send specific flowfiles to a specific remote MiNiFi Java. Testing with 1 MiNiFi works great, out from NiFi, into MiNiFi for processing. I'm prepping the MiNiFi flow on NiFi, templating, exporting, converting to yaml, modifying any security settings i need and restarting MiNiFi to accept the updated config.yml. Now i'm adding additional MiNiFis. On NiFi I've setup a PG with a RouteOnAttribute (Attribute is the specific MiNiFi name) to send to a specific S2S Output Port for that MiNiFi. Up to here, all seems fine. The problem i have is in the MiNiFi template. I have the RPG going into a processor, and that's when i specify the From Output Port which is specific to the MiNiFi. How can i template/generalize/variabilize/other the From Output so that i don't need to make separate template for each and every MiNiFi instance? I could theoretically move the RouteOnAttribute from the NiFi to the MiNiFi side but at that point it means i already sent a ton of data to all the MiNiFi instances instead of only to the one that's supposed to get it. I figured i can update the config.yml with the specific connections IDs, but that seems like a broken process that's very prone to mistakes and maintenance for every change i make to the flow. Does anyone have any pointers or tips for this situation?

benjbenj · ‎07-31-2021

An additional note for anyone coming here that isn't seeing the Generated Username/Password log entries (for example on Docker), you can set those by running bin/nifi.sh set-single-user-credentials <Username> <Password>.

Online	Offline
Last Visited	‎05-24-2023 01:11 AM

Member Since	‎07-31-2021 07:37 PM
Last Visited	‎05-24-2023 01:11 AM
Posts	6
Kudos received	2

Cloudera Community

Re: Sending data from NiFi to a specific MiNiFi Ja...

Re: Sending data from NiFi to a specific MiNiFi Ja...

Re: Sending data from NiFi to a specific MiNiFi Ja...

Re: Nifi Padding string ?

Sending data from NiFi to a specific MiNiFi Java

Re: Nifi 1.14 asks for username and password on Wi...