Support Questions
Find answers, ask questions, and share your expertise

Minifi-NiFi remote process group fails to connect

Explorer

Hi,

 

I have a minifi-0.5.0 configured as a service in a pi to send data to a nifi 1.9.2 in an aws node via 4G module.

 

I am observing that on almost 50% of the time that the service starts, data does not flow to nifi via a remote PG.

 

In the log the issue is shown as below:

 

2020-04-08 14:39:42,917 WARN [Timer-Driven Process Thread-1] o.a.nifi.remote.client.http.HttpClient Penalizing a peer Peer[url=<my URL>] due to java.io.IOException: Unexpected response code: 404 errCode:Abort errMessage:Unable to find port with id '01e59779-016e-1000-29f6-b28702b9d511'.
2020-04-08 14:39:42,919 INFO [Timer-Driven Process Thread-1] o.a.nifi.remote.client.http.HttpClient Couldn't find a valid peer to communicate with.

 

I can see that the issue is something to do with the input port of the RPG, configured in config.yml here: 

 

MiNiFi Config Version: 3
Flow Controller:
  name: <name>V0.02
  comment: ''
Core Properties:
  flow controller graceful shutdown period: 10 sec
  flow service write delay interval: 500 ms
  administrative yield duration: 30 sec
  bored yield duration: 10 millis
  max concurrent threads: 1
  variable registry properties: ''
FlowFile Repository:
  partitions: 256
  checkpoint interval: 2 mins
  always sync: false
  Swap:
    threshold: 20000
    in period: 5 sec
    in threads: 1
    out period: 5 sec
    out threads: 4
Content Repository:
  content claim max appendable size: 10 MB
  content claim max flow files: 100
  always sync: false
Provenance Repository:
  provenance rollover time: 1 min
  implementation: org.apache.nifi.provenance.MiNiFiPersistentProvenanceRepository
Component Status Repository: 
  buffer size: 1440
  snapshot frequency: 1 min
Security Properties:
  keystore: ''
  keystore type: ''
  keystore password: ''
  key password: ''
  truststore: ''
  truststore type: ''
  truststore password: ''
  ssl protocol: ''
  Sensitive Props:
    key:
    algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
    provider: BC
Processors:
- id: 1c28bf57-3237-39d3-0000-000000000000
  name: ListenUDP:5555
  class: org.apache.nifi.processors.standard.ListenUDP
  max concurrent tasks: 1
  scheduling strategy: TIMER_DRIVEN
  scheduling period: 0 sec
  penalization period: 30 sec
  yield period: 1 sec
  run duration nanos: 0
  auto-terminated relationships list: []
  Properties:
    Character Set: UTF-8
    Local Network Interface:
    Max Batch Size: '1'
    Max Size of Message Queue: '10000'
    Max Size of Socket Buffer: 1 MB
    Message Delimiter: \n
    Port: '5555'
    Receive Buffer Size: 65507 B
    Sending Host:
    Sending Host Port:
Controller Services: []
Process Groups: []
Input Ports: []
Output Ports: []
Funnels: []
Connections:
- id: ce64ed58-820f-3467-0000-000000000000
  name: ListenUDP:5555/success/01e59779-016e-1000-29f6-b28702b9d511
  source id: 1c28bf57-3237-39d3-0000-000000000000
  source relationship names:
  - success
  destination id: 01e59779-016e-1000-29f6-b28702b9d511
  max work queue size: 10000
  max work queue data size: 1 GB
  flowfile expiration: 0 sec
  queue prioritizer class: ''
Remote Process Groups:
- id: 56c5ff5a-aba7-3fbd-0000-000000000000
  name: ''
  url: <My URL>
  comment: ''
  timeout: 30 sec
  yield period: 10 sec
  transport protocol: HTTP
  proxy host: ''
  proxy port: ''
  proxy user: ''
  proxy password: ''
  local network interface: ''
  Input Ports:
  - id: 01e59779-016e-1000-29f6-b28702b9d511
    name: ship-data
    comment: ''
    max concurrent tasks: 1
    use compression: false
  Output Ports: []
NiFi Properties Overrides: {}

 

 

When the WARN occurs, it just keeps repeating the warning and never recovers. The pi itself has internet access - I can SSH in via WiFi and ping google for example. Restarting the minifi service fixes the issue, but not every time - I have to restart it a few times sometimes.

 

Could I please ask for some advice on this issue. TIA.