Configuring NiFi Site to Site protocol for AWS nodes with no public ip

[#1]

I have a Nifi cluster setup with 2 nodes in AWS EC2. One Nifi node has a public ip and 2nd one is on vpc. My cluster is up and running and I can see both my nodes from nifi public instance.

I want to add a remote process group pointing to same instance for load balancing. When I add a remote process group with same address as public server, it fails to communicate to node.

I have left nifi.remote.input.host as blank. Should I populate it with public ip address of server or private ip address of server ?

Thanks for any help.

[#2 · Accepted solution]

@Shishir Saxena

Your source NiFi will need to be able to communicate directly with both nodes in your target cluster in order to load-balance data. What version of NiFi are you using?

[#3]

HDF 2.0. Source nifi is also running on same instance. Looking at logs, it looks like nifi is trying to communicate on public ip address, but it can't communicate. How do I specify private ip for communication.

[#4]

The source NiFi will initially communicate with the target cluster over the same HTTP(s) port you would use to access the target NiFi cluster's UI. After that initial communication the target cluster will provide your source NiFi with the configured nifi.remote.input.host and nifi.remote.input.port for each node in teh target cluster along with teh current load on each node.

If you left the nifi.remote.input.host blank, Java will try to determine the hostname. This may result in either an internal hostname your source can not resolve or even just localhost. I highly recommend setting this property to a public facing FQDN for each node in your cluster.

Matt

[#5]

Thanks Matt. Source Nifi instance is not able to talk to target cluster because it it trying to access public ip that it can't access. I guess it is more of EC2 question, how can I get my node to communicate using public ip.

Only one of the node is public facing. Other node is not public facing and doesn't have a public facing FQDN.

[#6]

Thanks Matt for your help. I finally resolved this issue by referring to internal ip of hosts.