HDF 3.4 nifi new cluster self signed ssl enabled - unable to login nifi with cert "Insufficient Permissions"

[#1]

@MattWho 

unable to log in seems authorization issue whereas cert authentication is successful. for initial admin cert user.

HDF 3.4 nifi new cluster self-signed ssl enabled - unable to login nifi with cert "Insufficient Permissions"

Insufficient Permissions

• home

No applicable policies could be found. Contact the system administrator.

nifi-user.log:

2020-03-26 04:47:13,898 INFO [NiFi Web Server-17] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=nifiadmin, OU=NIFI) GET https://lpappnifixx:9091/nifi-api/flow/current-user (source ip: 72.191.44.165)
2020-03-26 04:47:13,899 INFO [NiFi Web Server-17] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=nifiadmin, OU=NIFI
2020-03-26 04:47:13,901 INFO [NiFi Web Server-17] o.a.n.w.a.c.AccessDeniedExceptionMapper identity[CN=nifiadmin, OU=NIFI], groups[] does not have permission to access the requested resource. No applicable policies could be found. Returning Forbidden response.

[#2]

Authorization.xml and user.xml are empty

cat authorizations.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizations>
<policies/>
</authorizations>

cat users.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
<groups/>
<users/>
</tenants>

[#3]

found quote in node identities, update and restarted the Nifi

now i am seeing different issue on node 1 though user.xml & auth.xml have data now with ssl cert intial admin 

cert generate as below :

sh /usr/hdf/current/nifi-toolkit/bin/tls-toolkit.sh standalone -B test12456@1234 -C 'CN=nifiadmin, OU=NIFI' -n 'lpappnifi002.node2.com,lpappnifi001.node1.com' --nifiDnPrefix 'CN=' --nifiDnSuffix ', OU=NIFI' -o /tmp/certs_ssl/ -K test12456@1234 -P test12456@1234 -S test12456@1234

ERROR: NIFI GUI from both UI's : Cannot replicate request to Node lpappnifi001.node1.com:8080 because the node is not connected

2020-03-26 07:14:48,604 INFO [Clustering Tasks Thread-2] o.a.n.c.c.ClusterProtocolHeartbeater Heartbeat created at 2020-03-26 07:14:48,469 and sent to lpappnifi001.node1.com:9088 at 2020-03-26 07:14:48,604; send took 134 millis
2020-03-26 07:14:50,892 INFO [Process Cluster Protocol Request-22] o.a.n.c.p.impl.SocketProtocolListener Finished processing request 09618070-4644-4e7e-b377-bb065453642d (type=HEARTBEAT, length=4880 bytes) from lpappnifi002.slower.ai:9091 in 128 millis
2020-03-26 07:14:53,740 INFO [Process Cluster Protocol Request-23] o.a.n.c.p.impl.SocketProtocolListener Finished processing request e4388aba-d0f1-4bcc-9fc4-11787f2ccea9 (type=HEARTBEAT, length=4881 bytes) from lpappnifi001.node1.com:9091 in 133 millis
2020-03-26 07:14:53,741 INFO [Clustering Tasks Thread-2] o.a.n.c.c.ClusterProtocolHeartbeater Heartbeat created at 2020-03-26 07:14:53,604 and sent to lpappnifi001.node1.com:9088 at 2020-03-26 07:14:53,741; send took 136 millis

Is this a cert issue? where the node 2 is unable to trust node 1?

[#4]

@MattWho 

i also noticed below error message from nifi-user.log

2020-03-26 14:48:14,119 INFO [NiFi Web Server-20] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: OpenId Connect is not configured.. Returning Conflict response.
2020-03-26 14:48:14,576 INFO [NiFi Web Server-16] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=nifiadmin, OU=NIFI) GET https://lpappnifi001.xxxx:9091/nifi-api/flow/current-user (source ip: 72.191.44.165)
2020-03-26 14:48:14,577 INFO [NiFi Web Server-16] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=nifiadmin, OU=NIFI
2020-03-26 14:48:14,586 INFO [NiFi Web Server-16] o.a.n.w.a.c.IllegalClusterStateExceptionMapper org.apache.nifi.cluster.manager.exception.IllegalClusterStateException: Cannot replicate request to Node lpappnifi001.xxx:8080 because the node is not connected. Returning Conflict response.

[#5 · Accepted solution]

@venk 

What you have run into at this point is a known issue.

Your cluster was originally setup and running unsecured over HTTP port 8080.

NiFi records the details of the nodes that are part of the cluster.  It does that so on later restarts it know that it should still be waiting on additional nodes to join before allowing users to make changes to the canvas.

The downside to this is that when you switched to being secured over HTTPS on port 9091, the cluster now thinks you should have twice the number of nodes as there really are.

But this is an easy fix.  Within your NiFi's conf directory you will find the file "state-management.xml". Inside that file you will find a section for NiFi's "local-provider" that will contain the directory where you can find your local state.  This path is normally the same on every node.

Shutdown your NiFi and go to this directory on every node in your cluster and delete the contents within that state directory.  Restart your NiFi and it will only create new entries for your secured nodes.

https://issues.apache.org/jira/browse/NIFI-7255

Hope this helps,

Matt

[#6]

exactly, for some reason though my nifi is 2 nodes secured cluster when I logged in it shows 4 nodes . two with secured and two with unsecured ports. stopped and followed the shared process. It came up clean.