Configuring Nifi processors redirects nifi-api url to internal ip

[#1]

I started running 3 nodes in the cluster without embedded zookeeper. I used a separate zookeeper instance which runs on one node and others connect to that first node. Nodes are shown as 3/3 connected. But there is something wrong. When I add a processor in one of the nodes, it gets added successfully and then configure that processor, the URL redirects to its internal ip.

For example, my public ip is : a.b.c.d and internal ip is a1.b1.c1.d1 then configuring processor redirects to :

a1.b1.c1.d1:9999/nifi-api/processors/ID_of_Processor

while it should be a.b.c.d/nifi-api/processors/ID_of_Processor

My properties are as follows:

nifi.web.http.host=FQDN_1
nifi.web.http.port=9999
nifi.remote.input.host=FQDN_1
nifi.remote.input.secure=false 
nifi.remote.input.socket.port=8888 
nifi.remote.input.http.enabled=true 
nifi.remote.input.http.transaction.ttl=30 sec

nifi.cluster.is.node=true 
nifi.cluster.node.address=FQDN_1 
nifi.cluster.node.protocol.port=9000 
nifi.cluster.node.protocol.threads=10 
nifi.cluster.node.event.history.size=25 
nifi.cluster.node.connection.timeout=5 sec 
nifi.cluster.node.read.timeout=5 sec 
nifi.cluster.firewall.file= 
nifi.cluster.flow.election.max.wait.time=2 mins 
nifi.cluster.flow.election.max.candidates=3 
nifi.zookeeper.connect.string=FQDN_1:2181 

nifi.zookeeper.connect.timeout=3 secs 
nifi.zookeeper.session.timeout=3 secs 
nifi.zookeeper.root.node=/nifi


where FQDN is in form of : ip-a-b-c-d.ec2.internal

[#2]

@Jatin Kheradiya

You have configured the nifi.properties file for your NiFi to use "FQDN_1". What does FQDN_1 resolve to on your NiFi instance? (public a.b.c.d or internal ip a1.b1.c1.d1)

What processor are you configuring and how have you configured it? Can you provide screenshot?

Thanks,

Matt

[#3]

Hello @Matt Clarke,

FQDN_1 is the fully qualified hostname which corresponds to internal ip.

Now I tried with making nifi.web.http.host=localhost, still it redirects one specific URL to internal IP.

This redirection is happening for all processors when I click on configure of a processor. I am not able to open the configuration part. I am able to click on some options of the processor like status, data provenance etc. But configure is redirecting to internal ip. Thus overall page shows following message:

Unable to communicate with NiFi
Please ensure the application is running and check the logs for any errors.
 

FQDN is: ip-a1-b1-c1-d1.ec2.internal

[#4 · Accepted solution]

@Jatin Kheradiya

Is there a proxy in front of the NiFi cluster? What is listening on the public IP and what is listening on the private IP?

[#5]

Hello @mgilman,

Yes in nginx, i configured it to proxy_pass but did not add the proxy headers. Now I added the headers and it is working fine.

Thanks.

[#6]

Awesome! Glad everything is working. Just to follow with extra details so others that come across this post will benefit...

There are a couple of key items to know when standing up NiFi behind a proxy.

1) NiFi is comprised of a number of web applications (web ui, web api, documentation, custom ui's, etc). So you'll need to set up your mapping to the root path. That way all context paths are pass through accordingly. For instance, if you only mapped the /nifi context path, the custom ui for Update Attributes will not work since it's available at /update-attribute-ui-<version>.

2) NiFi's rest api will generate uri's for each component on the graph. Since your coming through a proxy, you'll need to override certain elements of the uri's being generated. This is why your able to view the graph, but you cannot modify existing Processors. It's attempting to call back directly to your NiFi, not through your proxy. You can override the elements of the uri by adding the following HTTP headers when your proxy generates the HTTP request to the NiFi instance:

X-ProxyScheme - the scheme to use to connect to your proxy (https in this case)

X-ProxyHost - the host of your proxy

X-ProxyPort - the port your proxy is listening on

X-ProxyContextPath - the path you've configured to map to the NiFi instance