Support Questions
Find answers, ask questions, and share your expertise

NIFI Site to site -what am I missing ?

NIFI Site to site -what am I missing ?

Explorer

I'm trying to create a site to site connection between 2 Nifi servers. Nifi1 (Hostname wef2 - running on windows) need to deliver traffic to nifi2 (IP 10.168.233.168 - running on Linux) - I defined the connection and all is green but on Nifi1 I'm getting the following error messages

 

What am I missing? 

*port ID 0176100c-8d25-196b-1f72-6befa5cab12a is the input port in NIFI2

 

2020-12-01 01:54:13,508 INFO [pool-12-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 0 milliseconds
2020-12-01 01:54:20,677 INFO [Http Site-to-Site PeerSelector] o.apache.nifi.remote.client.PeerSelector Successfully refreshed peer status cache; remote group consists of 2 peers
2020-12-01 01:54:23,203 WARN [Timer-Driven Process Thread-3] o.a.nifi.remote.client.http.HttpClient Penalizing a peer Peer[url=http://localhost:8080/nifi-api] due to java.io.IOException: Unexpected response code: 404 errCode:Abort errMessage:Unable to find port with id '0176100c-8d25-196b-1f72-6befa5cab12a'.
2020-12-01 01:54:23,208 WARN [Timer-Driven Process Thread-3] o.a.nifi.remote.client.http.HttpClient Penalizing a peer Peer[url=http://wef2:8080/nifi-api] due to java.io.IOException: Unexpected response code: 404 errCode:Abort errMessage:Unable to find port with id '0176100c-8d25-196b-1f72-6befa5cab12a'.
2020-12-01 01:54:23,208 INFO [Timer-Driven Process Thread-3] o.a.nifi.remote.client.http.HttpClient Couldn't find a valid peer to communicate with.
2020-12-01 01:54:33,214 WARN [Timer-Driven Process Thread-2] o.a.nifi.remote.client.http.HttpClient Penalizing a peer Peer[url=http://wef2:8080/nifi-api] due to java.io.IOException: Unexpected response code: 404 errCode:Abort errMessage:Unable to find port with id '0176100c-8d25-196b-1f72-6befa5cab12a'.
2020-12-01 01:54:33,219 WARN [Timer-Driven Process Thread-2] o.a.nifi.remote.client.http.HttpClient Penalizing a peer Peer[url=http://localhost:8080/nifi-api] due to java.io.IOException: Unexpected response code: 404 errCode:Abort errMessage:Unable to find port with id '0176100c-8d25-196b-1f72-6befa5cab12a'.
2020-12-01 01:54:33,219 INFO [Timer-Driven Process Thread-2] o.a.nifi.remote.client.http.HttpClient Couldn't find a valid peer to communicate with.
2020-12-01 01:54:33,510 INFO [pool-12-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2020-12-01 01:54:33,510 INFO [pool-12-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 0 milliseconds** 

 

 

Nifi 1

 
 

Nifi 1.JPG

 

NIFI 2

 

Nifi 2.JPG

 

Remote processor 

remote proccesor.JPG

 

Remote process configuration

 

remote process configuration.JPG

3 REPLIES 3

Re: NIFI Site to site -what am I missing ?

Super Guru

Does 

0176100c-8d25-196b-1f72-6befa5cab12a

match the id for Input from NiFi 1

 

also right click to make sure you started.   you may need to do a refresh.

 

are both clusters or single nodes?

 

Restart both nodes. 

Re: NIFI Site to site -what am I missing ?

Super Guru

The other server needs to have site-site remote connections enabled.

 

https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.5.1/building-a-dataflow/content/configure-site-to-...

 

see

https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#site_to_site_properties

 

by default this is turned off

 

nifi.remote.input.host

nifi.remote.input.http.enabled

 

Re: NIFI Site to site -what am I missing ?

Master Guru

@dzbeda 

 

Seeing "localhost" in your shared log output leads to what may be the issue.

When you configure the URL in the Remote Process Group (RPG), it tells that NiFI RPG to communicate with that URL to fetch the Site-To-Site (S2S) details.  Included in those returned details are things like:
- number of nodes in target NiFi cluster (if standalone returns only one host)
- The hostnames of those node(s) (in this case it looks like maybe localhost is being returned
- Configured RAW port if configured
- Whether HTTP transport protocol is enabled
- etc...

So when you RPG actually tries to send FlowFiles over S2S it is trying to send to localhost which results in itself rather than the actual target linux NiFi it fetched the S2S details from.

When some properties are left unconfigured, NiFi returns whatever the OS resolves.
I am going to guess your linux server is returning localhost rather than the actual hostname.
You will want to verify your S2S configuration setup in the target NiFi (linux server):
http://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#site_to_site_properties

Try setting the "nifi.remote.input.host" to see if that helps you.

Hope this helps,
Matt