Using MiNiFi: Remote Process Group Connection Refused

[#1]

We set up our minifi & flow according to this tutorial: https://community.hortonworks.com/articles/56341/getting-started-with-minifi.html

I have done it several times to verify it was done correctly.

Within the server that minifi is installed on, upon starting it, after initialization, in the minifi-app.log, we get this error: p.p1

2017-04-04 08:59:27,307 ERROR [Timer-Driven Process Thread-1] o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=MinifiSource,target=http://foo.xyz.abc.com:8080/nifi] failed to communicate with http://foo.xyz.abc.com:8080/nifi due to java.net.ConnectException: Connection refused

We expected this was a port issue so we checked the ports using nmap -sT -O localhost on our Nifi Server and it seems the 10000 port is open. We also did a telnet on URL of our Nifi Server on our minifi server and it seemed to connect okay.

As a note: even when I just have a GenerateFlowFile -> RPG going to http://foo.xyz.abc.com:8080/nifi on my Nifi interface (essentially its remoting to itself), it still give me this same error as above.

Any guidance would be much appreciated.

[#2]

Nothing is really jumping out at me, it feels like something isn't configured correctly, but I'm not sure what.

Have you tried making nifi.remote.input.host the same value as the hostname you are using for the UI?

Just wondering since that was a 192 address and the web host looks like a hostname. The web host and s2s host can be different, but just wondering if there is some issue using the 192 address in this case.

[#3]

I am rather unsure what you mean.

nifi.remote.input.host is set to the same value as the hostname we are using for the Nifi UI, I think.

nifi.remote.input.host= 192.168.x.x

Hostname for UI= 192.168.x.x:8080/nifi

Do you mean to add the port and /nifi onto it in the nifi.properties folder?

[#4]

Sorry I may have just mis-understood the information...

In all of the logs it showed foo.xyz.abc.com and then for nifi.remote.input.host you said it was 192.168.x.x. I realize both are obfuscated, but I was assuming these were different values, otherwise I thought you would have put foo.xyz.abc.com for both of them.

If nifi.web.http.host and nifi.remote.input.host are the same in your nifi.properties, then you can ignore my previous comment.

[#5]

So I think perhaps theres some confusion.

foo.xyz.abc.com is the url of the location of the receiving server and of course, 8080/nifi is added to access the nifi interface

192.168.x.x is the IP address of the sending server. They are different values and should be.

In response to your comment: If nifi.web.http.host and nifi.remote.input.host are the same in your nifi.properties, then you can ignore my previous comment.

They are not the same - in fact, nifi.web.http.host is blank and when I put the same value as nifi.remote,input.host, the receiving servers Nifi UI does not work anymore (as expected)

It is interesting to note that I was actually able to get Nifi to talk to itself through localhost using these values

nifi.remote.input.host=localhost

nifi.remote.input.secure=false

nifi.remote.input.socket.port=10000

nifi.remote.input.http.enabled=true

nifi.remote.input.http.transaction.ttl=30 sec

I think my next step is to install Nifi on another server than my sending server and try to send to my receiving server to try to figure out which server is the problem child.

[#6]

Hey Bryan

That was it! If you check out the tutorial at https://community.hortonworks.com/articles/56341/getting-started-with-minifi.html

the wording makes it seem to me like we were meant to put in the IP address for the sending server. Thank you so much for your help. A week and half to figure this out.

[#7]

Ok nifi.remote.input.host should not be the value of a remote server, it should either be blank just like the web host which means it will bind to all interfaces, or it should be a specific hostname/ip of the current node that you are editing nifi.properties on.

Here is how it works, lets say you have a standalone NiFi instance with a RPG trying to send data to a 3 node cluster... In the RPG you put the URL of the target cluster's UI like http://somehost:8080/nifi, the RPG then makes a REST call to http://somehost:8080/nifi to ask it for the information about all the nodes in the cluster, the response contains the value of nifi.remote.input.host and nifi.remote.input.socket.port for each of the three nodes in the destination cluster, so now the source instance knows where to send data two.

[#8]

Switch to HTTP, make sure everything has permissions. check firewalls. I have had wrong input names sometimes in the config.yml check those.