question Re: Enabling SSL in NIFI Cluster in Archives of Support Questions (Read Only)

Enabling SSL in NIFI Cluster

Jasthi — Thu, 03 May 2018 20:44:02 GMT

I was trying to enable SSL in NIFI cluster which gives an error at NIFI UI :

Insufficient Permissions home

Untrusted proxy CN=nifiadmin, OU=NIFIrsdevhdf2.medassurant.local, OU=NIFI

I added a picture of My NIFI configuration for SSL enabling :nifi-config.png

Re: Enabling SSL in NIFI Cluster

MattWho — Thu, 03 May 2018 21:19:20 GMT

@Veerendra Nath Jasthi

Where did you get the keystore files you are using on each of your nodes from?

I suggest performing a verbose listing on your keystore ( keytool -v -list -keysrtore <keystore,jks file> )

That listing should show a single "PrivateKeyEntry" and that should then show a "Owner" and "Issuer" as below exmaple does:

Alias name: nifi-key
Creation date: Apr 19, 2018
Entry type: PrivateKeyEntry
Certificate chain length: 2
Certificate[1]:
Owner: CN=nifi-sme-15.openstacklocal, OU=NIFI
Issuer: CN=nifi-sme-26.openstacklocal, OU=NIFI
Serial number: 162df02fcaf00000000
Valid from: Thu Apr 19 17:45:37 UTC 2018 until: Sun Apr 18 17:45:37 UTC 2021
Certificate fingerprints:
 MD5:  B2:B3:A8:D0:DC:E4:98:1F:53:30:A6:B4:E0:79:41:1A
 SHA1: 04:D9:3A:84:7B:75:AE:90:DD:C9:41:D3:83:1C:4F:BB:3C:18:EC:FA
 SHA256: AD:69:23:80:A1:06:1A:6C:32:A4:4C:95:B5:0E:5F:0E:AA:12:BE:DF:05:84:B8:53:27:F3:D9:46:DD:89:03:7A
 Signature algorithm name: SHA256withRSA
 Version: 3
Extensions:
#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: A0 9D B4 20 80 B3 6D 31   70 2E 73 B0 7E E0 17 F9  ... ..m1p.s.....
0010: 3D 31 A1 B4                                        =1..
]
]
#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:false
  PathLen: undefined
]
#3: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  clientAuth
  serverAuth
]
#4: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Non_repudiation
  Key_Encipherment
  Data_Encipherment
  Key_Agreement
]
#5: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: nifi-sme-15.openstacklocal
  DNSName: nifi-sme-15.openstacklocal
]
#6: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 9B 43 A1 B1 3D 9B AF B4   1B 1B 8F DA 31 D2 14 88  .C..=.......1...
0010: 4E 3E 93 A9                                        N>..
]
]

The "Owner" DN form each of your Nodes keystores should match the node identities you entered in your configs (case sensitive).
-
Also note that you have names every one of your entries as "Node Identity 1". You should instead have a unique number for each node identity.

My guess here is that maybe your keystore contains more then one "PrivateKeyEntry". Did you create a user certificate "CN=nifiadmin, OU=NIFIrsdevhdf2.medassurant.local, OU=NIFI" and import in to each of your nodes keystores?

NiFi has no way to be configured to select a specific "PrivateKeyEntry" when multiple exist in same keystore.
-
The keystore should contain only 1 "PrivateKeyEntry". It may contain many "trustedCertEntry" entries.

Commonly your keystore.jks will contain only the single PrivateKeyEntry and your truststore.jks will contain 1 to many "TrustedCertEntry".

Once you have made the necessary corrections to your keystore.jks file and/or node identity configurations, you will need to delete the users.xml and authorizations.xml files that NiFi created as they are only created once. If they already exist, they will not be updated by changes you make to node identity configurations or initial admin identities. Once you can successfully access the secured NIFi UI as your initial admin, you will add the rest of your users and se their policies directly from within the UI.

Thanks,

Matt

If you found this answer addressed your question, please take a moment to login to the forum and click "accept" on the answer.

Re: Enabling SSL in NIFI Cluster

Jasthi — Thu, 03 May 2018 21:44:46 GMT

Hi @Matt Clarke Thanks for quick response and while I was trying to keytool -v -list -keystore keystore.jks this I was getting below error:

keytool error: java.io.IOException: Keystore was tampered with, or password was incorrect

java.io.IOException: Keystore was tampered with, or password was incorrect

and also I am new to this SSl Concept Could you please explain me the resolution little bit clear ?

Re: Enabling SSL in NIFI Cluster

MattWho — Thu, 03 May 2018 22:02:17 GMT

@Veerendra Nath Jasthi

*** Forum tip: Try to avoid responding to an existing "answer" by starting a new answer. It makes following the conversation very hard. Instead use "Add comment" on the existing answer.

When running the keytool command, juts try hitting enter when prompted for password without entering anything.
-
Did you use the include NIFi CA to create your keystore and truststore?
Did you use the NiFi TLS-toolkit to generate your user certificate?
-

Thanks,

Matt

Re: Enabling SSL in NIFI Cluster

Jasthi — Thu, 03 May 2018 22:14:14 GMT

I have used NiFi TLS-toolkit & Below are outputs from Nodes:

Node1:

Entry type: PrivateKeyEntry

Certificate chain length: 2

Certificate[1]:

Owner: CN=nifiadmin, OU=NIFIrsdevhdf1.medassurant.local, OU=NIFI

Issuer: CN=rsdevhdf3.medassurant.local, OU=NIFI

Node2:

Entry type: PrivateKeyEntry

Certificate chain length: 2

Certificate[1]:

Owner: CN=nifiadmin, OU=NIFIrsdevhdf2.medassurant.local, OU=NIFI

Issuer: CN=rsdevhdf3.medassurant.local, OU=NIFI

Node3:

Entry type: PrivateKeyEntry

Certificate chain length: 2

Certificate[1]:

Owner: CN=nifiadmin, OU=NIFIrsdevhdf3.medassurant.local, OU=NIFI

Issuer: CN=rsdevhdf3.medassurant.local, OU=NIFI

Re: Enabling SSL in NIFI Cluster

MattWho — Thu, 03 May 2018 23:25:51 GMT

@Veerendra Nath Jasthi

The "Owner" DN typically has a CN that matches the hostname of the server on which the certificate is being used. In this case it looks like you create a "nifiadmin" certificate on each node. If your CN in the "owner" DN does not match your servers hostname, you will need to have a SubjectAlternativeName (SAN) in your certificate that does.

Bottom line you will likely need new certificates here.
-

Then make sure that each of those server DNs match what you provided as DNs for the "Node Identity 1=, Node Identity 2=, Node Identity 3=".

Also, do not forget to delete the users.xml and authorizations.xml files so they get re-created with correct entries.

Thanks,
Matt

Re: Enabling SSL in NIFI Cluster

Jasthi — Fri, 04 May 2018 02:22:57 GMT

I have followed the above steps to generate new certificate and now ending up with the error like :

Insufficient Permissions

Unknown user with identity 'CN=nifiadmin, OU=NIFI'. Contact the system administrator.

Re: Enabling SSL in NIFI Cluster

MattWho — Fri, 04 May 2018 04:03:16 GMT

@Veerendra Nath Jasthi

You are almost there. It sounds like you may have all the certificates in place fo the NiFi cluster itself to work correctly. Any user wishing to access a secured NiFi must successfully be authenticated and authorized for the specific NiFi resources they need/want access to.

The DN you provided as the "Initial Admin Identity" needs to mach the DN exactly that is coming from the user/client. By default NiFi expects that users present a TLS certificate by which authentication is verified. NiFi can also be configured to support kerberos and LDAP authentication methods (see login-identity-providers.xml file).

Check the nifi-user.log to get more specific details on the permission denied you are seeing. I also see that appears to match exactly with what you had configured as your Initial Admin Identity in the attached screenshot, so make sure it is correct in your user.xml file (I am assuming you did delete these files on all nodes before you restarted with new configurations in Ambari).

If you want to share what is in your nifi-user.log when you try to access the URL, that may help determine what is still not correct.

Thanks,

Matt

Re: Enabling SSL in NIFI Cluster

mustafa_qizilba — Fri, 04 May 2018 08:28:30 GMT

Confirm below are done.

Encrypt Configuration Master Key Password: <password, min 12, character and numbers are must>

Sensitive property values encryption password: <password, min 10, character and numbers are must>

NiFi CA Token: <password>

Click the check box and Enable SSL

nifi.initial.admin.identity = admin

Re: Enabling SSL in NIFI Cluster

Jasthi — Fri, 04 May 2018 20:57:24 GMT

new-nifi-ssl-config.png@Matt Clarke Below is the nifi-user.log & attached new SSl config in ambari :

2018-05-04 09:49:04,844 INFO [NiFi Web Server-22] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response.

2018-05-04 09:49:04,877 INFO [NiFi Web Server-1237] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=nifiadmin, OU=NIFI) GET https://rsdevhdf3.medassurant.local:9091/nifi-api/flow/current-user (source ip: 10.5.43.133)

2018-05-04 09:49:04,880 INFO [NiFi Web Server-1237] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=nifiadmin, OU=NIFI

2018-05-04 09:49:04,897 INFO [NiFi Web Server-1237] o.a.n.w.a.c.AccessDeniedExceptionMapper CN=nifiadmin, OU=NIFI does not have permission to access the requested resource. Unknown user with identity 'CN=nifiadmin, OU=NIFI'. Returning Forbidden response.

Re: Enabling SSL in NIFI Cluster

MattWho — Fri, 04 May 2018 21:57:22 GMT

@Veerendra Nath Jasthi

So nifi-user.log shows that authentication was successful for your user "CN=nifiadmin, OU=NIFI". This puts the issue squarely on the authorization side of things. Authorization configurations are in the authorizers.xml file.
-
Since it sounds like you are using the default file based authorization provider, you will want to inspect what is in your users.xml and authorizations.xml files NiFi generated.

What do you have in your users.xml?

You should find an entry for "CN=nifiadmin, OU=NIFI" in there associated to a unique UUID. That UUID is then used to associate that user to various access policies in the authorizations.xml file. Be mindful that NiFi is case sensitive and blank spaces are valid characters including leading and trailing whitespace. "CN=nifiadmin, OU=NIFI" is not equal to "CN=nifiadmin, OU=NIFI " (trailing white space) to NiFi.

Also confirm what URL you are trying to access? (https://<nifinode-hostname>:9091/nifi )

Thanks,

Matt

Re: Enabling SSL in NIFI Cluster

MattWho — Fri, 04 May 2018 22:30:47 GMT

@Veerendra Nath Jasthi

You can see in your users.xml file that the usr identity does not match exactly with what is in your nifi-user.log. They must match exactly.

<user identifier="49527e2e-41db-3e98-9926-49021fd68a56" identity="CN=nifiadmin,OU="/>

while user.log has:

CN=nifiadmin, OU=NIFI

Assuming you had above set as your Initial Admin Identity in Ambari NiFi configs and you deleted the users.xml and authorizations.xml files on all nodes before staring NIFi via Ambari, new users.xml and authorizations.xml files should have been generated correctly.

So you have two options here:
1. Stop NiFi and manually edit the users.xml file on every node so that the identity matches exactly and restart NiFi on all nodes.

2. Stop NiFi via Ambari, verify "Initial Admin Identity" NIFi property is set correctly, delete the users.xml and authorizations.xml on all nodes, and teh start NiFi via Ambari. Lates configs will be written to NiFi config files and NiFi will create new users.xml and authorizations.xml files on each node on startup.

Thanks,

Matt

Re: Enabling SSL in NIFI Cluster

Jasthi — Fri, 04 May 2018 22:34:23 GMT

user-xml.png

Re: Enabling SSL in NIFI Cluster

MattWho — Fri, 04 May 2018 22:46:53 GMT

@Veerendra Nath Jasthi

Your latest two screenshots confirm what is suspected above. You did not have the correct/full DN configured as your "Initial Admin Identity". The corrective actions I provided in previous response should get you squared away here.

Re: Enabling SSL in NIFI Cluster

Jasthi — Fri, 04 May 2018 23:09:57 GMT

@Matt Clarke I tried the changes you mentioned in the above post but still no luck.Here I have attached 3 screen shots are:

user-xml.png

ambari-config.png

nifi-user-log.png

Re: Enabling SSL in NIFI Cluster

MattWho — Fri, 04 May 2018 23:16:41 GMT

@Veerendra Nath Jasthi

You are so very very close. Remember how i commented above that the DN string must match exactly?

Your admin DN is:

CN=nifiadmin, OU=NIFI

But you entered the following as your Initial Admin Identity:

CN=nifiadmin,OU=NIFI

Note that you are missing the space between "CN=nifiadmin," and "OU=NIFI"

Thanks,

Matt

Re: Enabling SSL in NIFI Cluster

Jasthi — Sat, 05 May 2018 00:48:36 GMT

After changes made as above now error:

Insufficient Permission

Untrusted proxy CN=nifiadmin, OU=NIFIrsdevhdf1.medassurant.local, OU=NIFI

user-log.png

and user.xml remains same like before:

<users>

</users>

</tenants>

Re: Enabling SSL in NIFI Cluster

MattWho — Sat, 05 May 2018 01:50:48 GMT

@Veerendra Nath Jasthi

The DN there is coming from the keystore being used by your NiFi nodes. I have no idea why the certs created for your servers all have nifiadmin in them....
...

But just like your user DN, the node identities must match exactly with what is in those server certs in the keystore..

<property name="Node Identity 1">CN=nifiadmin, OU=NIFIrsdevhdf1.medassurant.local, OU=NIFI</property>
<property name="Node Identity 2">CN=nifiadmin, OU=NIFIrsdevhdf2.medassurant.local, OU=NIFI/</property>
<property name="Node Identity 3">CN=nifiadmin, OU=NIFIrsdevhdf3.medassurant.local, OU=NIFI</property>

so you will need to edit your node identities so they match the above and once again stop NiFi, remove your users.xml and authorizations.xml files, and then start NiFi again via Ambari.

Thank you,
Matt