Skip to main content
Meeran
Meeran
Silver ⭐️⭐️⭐️
March 30, 2023
Solved

NiFi Cluster but single point of URL

Hello All,

 

I have configured nifi flow with three node with external zookeeper

node1.domain.com

node2.domain.com

node3.domain.com

 

with three zookeeper nodes

zk1.domain.com

zk2.domain.com

zk3.domain.com

 

Everything is working fine but the requirement here is my flow has trigger points from external client systems to whom I have give a common URL like

node.domain.com to avoid any changes on the client teams if any of nodes goes down.

 

Any idea how to configure this.

MattWho
MattWhoAnswer
Titanium ⭐️⭐️⭐️
April 6, 2023

@Meeran 

Did you setup sticky sessions in your load balancer?
When you provide your ldap username and password, a JWT Token issue to the client which is only valid for use with the specific NiFi node that issued it.  So if you enter your username and password and you LB sends next request to a different node to load the flow, the JWT token will not be accepted to identify your user on that other node.

Watch the nifi-user.log on all nodes when you attempt to login.  In one of those logs you will see authentication success logged for your user.  I suspect that then the above exception about  "anonymous" is then in a different node's user log.

If you are having trouble configuring your load balancer, it would be best to start a new community question for that query so it gets to a wider audience.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

 

View in thread

7 replies

MattWho
Titanium ⭐️⭐️⭐️
MattWho
Titanium ⭐️⭐️⭐️
March 30, 2023

@Meeran 
The solution here is to setup an external load balancer in front of your 3 NiFi nodes.  Then have your clients point at that load balancer.  The Load Balancer would then be responsible for sending your client request to one of the available NiFi nodes. (If a node goes down, the Load Balancer simply does not send client requests to that node.

When using a Load Balancer in front of your NiFi cluster, it is important that the Load Balancer is configured to use sticky sessions.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt

Translate
    Meeran
    MeeranAuthor
    Silver ⭐️⭐️⭐️
    April 2, 2023

    I have configured this and set the 
    nifi.web.proxy.host: node1.domain.com:8443,node2.domain.com:8443,node3.domain.com:8443,loabalancerUrl.domain.com:8443

    but while logging in I see below issue. I have checked the logs but no errors in nifi-user.logs

    Meeran_0-1680421347163.png

     

    Translate
    Meeran
    MeeranAuthor
    Silver ⭐️⭐️⭐️
    March 31, 2023

    Thanks for the reply @MattWho I have implemented in the same way what you have said but while accessing with the load balancer url I am getting the below exception

     

    System Error
    The request contained an invalid host header [internal-myappname-13718787520.us-east-1.elb.amazonaws.com:8443] in the request [/]. Check for request manipulation or third-party intercept.

     

    Translate
    MattWho
    Titanium ⭐️⭐️⭐️
    MattWho
    Titanium ⭐️⭐️⭐️
    April 4, 2023

    @Meeran Where are you seeing above exception?

    System Error
    The request contained an invalid host header [internal-myappname-13718787520.us-east-1.elb.amazonaws.com:8443] in the request [/]. Check for request manipulation or third-party intercept.

    In your Load Balancer logs?
    This host header "internal-myappname-13718787520.us-east-1.elb.amazonaws.com" is in your nifi.web.proxy.host list or in the SAN list within each of your node's certificates?

    In NiFi, check the nifi-user.log and nifi-app.log for any WARN or ERROR log output being produced when you try to access NiFi via the load balancer URL.

    Translate
    Meeran
    MeeranAuthor
    Silver ⭐️⭐️⭐️
    April 6, 2023

    Hello @MattWho 

     

    Load balancer started showing the login page after setting below properties in nifi.properties

    nifi.web.proxy.context.path=/
    nifi.web.proxy.host=myapp.dev.domain.com:8443  (load balancer url)

     

    but after giving the logins details nifi canvas is not coming up

    I am seeing the below error while access via load balancer URL in nifi-user.log file

     

    2023-04-06 15:04:35,861 INFO [NiFi Web Server-17] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 10.1xx.xx.xxx [<anonymous>] GET https://myapp.dev.domain.com:8443/nifi-api/flow/current-user
    2023-04-06 15:04:35,862 WARN [NiFi Web Server-17] o.a.n.w.s.NiFiAuthenticationFilter

    Authentication Failed 10.1xx.xx.xxx GET https://myapp.dev.domain.com:8443/nifi-api/flow/current-user [Anonymous authentication has not been configured.]

     

    I am using LDAP for authentication.

     

    Do I need to change any other properties/settings, please help

    Translate
    Terms & ConditionsAccessibility statement