question Two NiFi Instances/Clusters Communication Timeout Issue in Archives of Support Questions (Read Only)

Two NiFi Instances/Clusters Communication Timeout Issue

alvinuw — Thu, 11 Aug 2016 21:46:40 GMT

I have two NiFi instances deployed in AWS. I'd like to test the RGP across clusters

US-East: 172.31.48.1

Dataflow: RGP->PutFile

# Site to Site properties

nifi.remote.input.socket.host=172.31.48.1

nifi.remote.input.socket.port=9090

nifi.remote.input.secure=false

# web properties #

nifi.web.http.host=172.31.48.1

nifi.web.http.port=8080

Singapore: 172.31.11.2

Dataflow: GetFile->output_port

# Site to Site properties

nifi.remote.input.socket.host=172.31.11.2

nifi.remote.input.socket.port=9090

nifi.remote.input.secure=false

# web properties #

nifi.web.http.host=172.31.11.2

nifi.web.http.port=8080

The GetFile processor can fetch files and queue them in Singapore instance.

The RGP can successfully detect the "output_port" by setting the url as "http://sig-public-ip:8080/nifi" in US-east instance.

However, RGP log shows :

14:38:14 UTC ERROR e93c67c6-3472-44e4-aa3e-78613ce37e47 us-ease-public-ip:8080

RemoteGroupPort[name=output_port,target=http://sig-public-ip:8080/nifi] failed to communicate with http://sig-public-ip:8080/nifi due to java.net.SocketTimeoutException

When I test the network connection,

ubuntu@ip-172-31-48-1:/opt$ telnet sig-public-ip 9090 Trying sig-public-ip... Connected to sig-public-ip.

ubuntu@ip-172-31-11-2:/opt$ telnet us-ease-public-ip 9090 Trying us-ease-public-ip... Connected to us-ease-public-ip.

The same for port 8080.

Any settings I should do to enable the RPG communication?

Thanks.

Re: Two NiFi Instances/Clusters Communication Timeout Issue

mpandit — Fri, 12 Aug 2016 02:43:19 GMT

Can you please make sure to disable the firewall on the nifi node.( service iptables stop) and try it again.

Re: Two NiFi Instances/Clusters Communication Timeout Issue

alvinuw — Fri, 12 Aug 2016 03:09:56 GMT

Hi @milind pandit

I deployed in an AWS instance. It seems iptables is not started.

ubuntu@ip-172-31-xx-xx:~$ sudo service iptables stop

iptables: unrecognized service

Telnet works on both 8080 and 9090 ports. Output port can be detected by RPG, but data can't be transferred.

Thanks.

Re: Two NiFi Instances/Clusters Communication Timeout Issue

mpandit — Fri, 12 Aug 2016 03:39:55 GMT

can you please provide the additional logs by setting <logger name="org.apache.nifi.remote" level="DEBUG"/> in your logback.xml.

Re: Two NiFi Instances/Clusters Communication Timeout Issue

alvinuw — Sat, 13 Aug 2016 00:05:34 GMT

Figured out it.

It's not the firewall issue.

Since the cluster and edge node are deployed in AWS.

The remote host of edge node for site-to-site should be set as its public ip, rather than private ip.

Singapore

Dataflow: GetFile->output_port

# Site to Site properties

nifi.remote.input.socket.host=sig-public-ip