Hi,   I have a minifi-0.5.0 configured as a service in a pi to send data to a nifi  1.9.2 in an aws node via 4G module.   I am observing that on almost 50% of the time that the service starts, data does not flow to nifi via a remote PG.   In the log the issue is shown as below:   2020-04-08 14:39:42,917 WARN [Timer-Driven Process Thread-1] o.a.nifi.remote.client.http.HttpClient Penalizing a peer Peer[url=<my URL>] due to java.io.IOException: Unexpected response code: 404 errCode:Abort errMessage:Unable to find port with id '01e59779-016e-1000-29f6-b28702b9d511'. 2020-04-08 14:39:42,919 INFO [Timer-Driven Process Thread-1] o.a.nifi.remote.client.http.HttpClient Couldn't find a valid peer to communicate with.   I can see that the issue is something to do with the input port of the RPG, configured in config.yml here:    MiNiFi Config Version: 3 Flow Controller: name: <name>V0.02 comment: '' Core Properties: flow controller graceful shutdown period: 10 sec flow service write delay interval: 500 ms administrative yield duration: 30 sec bored yield duration: 10 millis max concurrent threads: 1 variable registry properties: '' FlowFile Repository: partitions: 256 checkpoint interval: 2 mins always sync: false Swap: threshold: 20000 in period: 5 sec in threads: 1 out period: 5 sec out threads: 4 Content Repository: content claim max appendable size: 10 MB content claim max flow files: 100 always sync: false Provenance Repository: provenance rollover time: 1 min implementation: org.apache.nifi.provenance.MiNiFiPersistentProvenanceRepository Component Status Repository: buffer size: 1440 snapshot frequency: 1 min Security Properties: keystore: '' keystore type: '' keystore password: '' key password: '' truststore: '' truststore type: '' truststore password: '' ssl protocol: '' Sensitive Props: key: algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL provider: BC Processors: - id: 1c28bf57-3237-39d3-0000-000000000000 name: ListenUDP:5555 class: org.apache.nifi.processors.standard.ListenUDP max concurrent tasks: 1 scheduling strategy: TIMER_DRIVEN scheduling period: 0 sec penalization period: 30 sec yield period: 1 sec run duration nanos: 0 auto-terminated relationships list: [] Properties: Character Set: UTF-8 Local Network Interface: Max Batch Size: '1' Max Size of Message Queue: '10000' Max Size of Socket Buffer: 1 MB Message Delimiter: \n Port: '5555' Receive Buffer Size: 65507 B Sending Host: Sending Host Port: Controller Services: [] Process Groups: [] Input Ports: [] Output Ports: [] Funnels: [] Connections: - id: ce64ed58-820f-3467-0000-000000000000 name: ListenUDP:5555/success/01e59779-016e-1000-29f6-b28702b9d511 source id: 1c28bf57-3237-39d3-0000-000000000000 source relationship names: - success destination id: 01e59779-016e-1000-29f6-b28702b9d511 max work queue size: 10000 max work queue data size: 1 GB flowfile expiration: 0 sec queue prioritizer class: '' Remote Process Groups: - id: 56c5ff5a-aba7-3fbd-0000-000000000000 name: '' url: <My URL> comment: '' timeout: 30 sec yield period: 10 sec transport protocol: HTTP proxy host: '' proxy port: '' proxy user: '' proxy password: '' local network interface: '' Input Ports: - id: 01e59779-016e-1000-29f6-b28702b9d511 name: ship-data comment: '' max concurrent tasks: 1 use compression: false Output Ports: [] NiFi Properties Overrides: {}     When the WARN occurs, it just keeps repeating the warning and never recovers. The pi itself has internet access - I can SSH in via WiFi and ping google for example. Restarting the minifi service fixes the issue, but not every time - I have to restart it a few times sometimes.   Could I please ask for some advice on this issue. TIA. ... View more

Hi, Running NiFi 1.7.1 on Ubuntu 18.04. I have a nifi instance on a physical laptop on one site sending SCADA and local resource monitoring data via S2S to another nifi instance in AWS. Every day I have the laptop rebooting using cronjob. I have noticed that the queue size drops about 10MB when the reboot occurs. I thought nifi keeps the contents of the queue even when rebooted. Can I please ask for advice on how I can diagnose this, and to make sure there is no data being lost. I am trending the data on the AWS side and there doesn't seem to be any breaks... Below are trends of the resource monitoring on the laptop. The reboot occurs at 0400. Thank you ... View more

Hi, NIFI 1.4.0/1.5.0. Is there a difference in what gets sent via Ambari Reporting Task and Site to Site Reporting tasks? Thanks ... View more

Hello, I have multiple NiFi instances in separate AWS EC2 instances. I'd like to monitor them all in one dashboard or location. As these guys were not deployed by Ambari, that option may not be the best. Could I please request suggestions on how I should monitor the instances? Thank you ... View more

Hi there, I am using a ListenTCP to capture data coming from an Arduino on a TCP port over WiFi. When there is a loss of comms over wifi for various reasons (loss of signal, power cut), the Arduino is programmed to attempt to connect back when connection is restored. I am seeing at times the max connections warning as the arduino attemps to reconnect, and it is unable to regain data flow for a long time - it seems to be a random amount of time from a few minutes up to half an hour. I have had a look at this article https://community.hortonworks.com/questions/90478/how-to-release-occupied-connections-in-listentcp-p.html which suggests that the Arduino side should close the connection, however during an abrupt connection loss, NiFi would not be able to detect a closure. Does anybody know if there is a set timeout period on the max connections condition? Is there a way to control how the max connection warning is triggered? Thank you ... View more

Hi, I am trying to set up two NiFis (v1.4.0) on separate AWS EC2 instances and have them communicate securely. I have secured each NiFi so you need SSL authentication to access the UI on a browser. However I am not able to have the two instances communicate S2S. I have had success with UNSECURED NiFi instances with them communicating on RAW protocol. However I don't think it's secure so would like to lock down the S2S comms, as well as NiFi UI access as well. I'm pretty sure the properties nifi.remote.input.* and nifi.web.* are the ones to modify, but am confused as I can't seem find an example on the web for securing both the UI browser access and S2S comms. Could somebody please point me in the right direction? Thank you ... View more