NiFi 1.5 System Error: Invalid host header

[#1]

Hi Guys,

I just upgraded NiFi from v1.4 to v1.5. The cluster is set in secure mode and running in Kubernetes.

But I found below error in Web UI.

System Error The request contained an invalid host header [server:port] in the request [/nifi/]. Check for request manipulation or third-party intercept.

Is it due to nifi.web.proxy.host property? How do I setup it? the IP of the nifi node?

Thanks.

[#2]

@Alvin Jin

With NiFi 1.5, NiFi has become more restrictive with requard to the allowed headers coming from a client.

The hostname in the request header is checked against the configured hostname in the nifi.properties file:

nifi.web.http(s).host=

If they do not match, you will encounter the error you are seeing.

So you will need to access the NiFi UI in 1.5 using the same hostname as specified in that property.

Thanks,

Matt

[#3]

Hi @Matt Clarke,

Thank you for your response.

Is it nifi.web.http(s).host or nifi.web.proxy.host?

I have already put nifi.web.https.host = the-nifi-node-ip

We deploy nifi cluster in Kubernete, What should I change it to?

Thanks.

[#4]

@Alvin Jin

NiFi would have never started if a port was not specified. Let me be clear that you are setting up yoru NiFi to either run over a secure or non-secure port and not both.

for a non-secure NiFi set these:

nifi.web.http.host=
nifi.web.http.port=

and for a secured NiFi, set these:

nifi.web.https.host= 
nifi.web.https.port=

I am not familiar with Kubernete containers. Is the hostname of the container the same as the public facing hostname?

Thank you,

Matt

[#5]

Hi @Matt Clarke,

Thank you for your clarification. Yes, I use the secure mode.

In NiFi v1.4, I setup the two properties as below.

nifi.web.https.host= nifi-0.svc

nifi.web.https.port= 9443

Please note nifi-0.svc is the pod name of the first nifi node.

Then, we have nifi-1.svc, nifi-2.svc in the setting of the other nifi nodes.

Normally, if we want to access NiFi cluster outside of Kubernetes,

we have to use the master node ip of Kubernetes. e.g. https://master-ip:9443/nifi/

Above worked perfectly before.

Now my question is in NiFi v1.5. What should I set for nifi.web.https.host?

1. If I don't do any changes, it gives me "Invalid Host Header" error.

2. If I set it as nifi.web.https.host = master-ip in all three nifi nodes in the cluster.

The error in log is:

2018-01-18 17:10:28,017 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: Cannot assign requested address

One thing confuses me is: I have three nifi nodes in a cluster. Should I set "nifi.web.https.host" in three nifi.property file to the same ip value?

BTW, are we able to disable this request header check?

Thanks.

[#6]

@Alvin Jin

Looks like you are hitting https://issues.apache.org/jira/browse/NIFI-4761

If you add a local host entry on your box that resolved "nifi-0.svc" to the IP associated to "master-ip", that should work if putting https://nifi-0.svc:9443/nifi/ does not work already.

Is that master node ip the same ip that is assigned to nifi-0.svc? If so, try setting the nifi.web.https.host= to the IP instead of the hostname.

Thanks,

Matt

[#7]

@Alvin Jin

The request header check cannot be disabled.

[#8]

Hi @Matt Clarke,

Kubernetes master ip is different with nifi-0.svc, which is an external pod dns.

So nifi-0.svc can't been access outside of Kubernetes.

I feel that nifi.web.proxy.host may work for this case.

Is there a way to disable the Request Header Check?

Thanks.

[#9]

The issue is that NiFi's Jetty server is being more restrictive about only accepting requests that were intended for its actual host.

So in your scenario, NiFi's Jetty server is listening on nifi-0.svc... then when you access https://master-ip:9443/nifi in your browser, the host header is set to master-ip, so when NiFi's Jetty server gets the request it rejects it because master-ip is not nifi-0.svc.

NIFI-4671 that Matt pointed out is going to add some way to white-list hostnames that should not be rejected, so you would just put master-ip in the white-list and it would let it through.

In the meantime, does Kubernetes have a way to control the headers that get forwarded from the master IP to the NiFi server?

For example, if someone had a load balancer in front of NiFi they would access the UI through the load balancer host which would create this same problem, but if you tell the load balancer to strip off the host header then it won't happen.

I don't know anything about Kubernetes so I have no idea if this is possible.

[#10]

NiFi 1.5 is validating "Host Header" and in Kubernetes the pod name is equal to host name. Unfortunately podname is generated during runtime and it is difficult to set in Host header.

Hortworks team, can you guys please help here to disable Host header check or work around of this issue.

,

It seems the check of "Host Header" of the request cannot be disabled. NiFi is expecting "pod" name of kubernetes as hostname in Host header. Unfortunately, It is not recommended to create "pod" explicitly it is created by "Deployment" and name of pod will be randomly generated during deployment. Is it possible for NiFi development team to provide some thought so that we can deploy in Kubernetes environment.