Support Questions

Find answers, ask questions, and share your expertise

How to Generate Certs to secure 3 node nifi cluster and configs

avatar
Explorer

How to Generate Certs to secure 3 node nifi cluster and configs

 

All, I am new to nifi and started to secure nifi 

can some share how to generate certs using toolkit standalone

 

Do I need to generate certs on all nodes

 

node1.xyz.com

node2.xyz.com

node3.xyz.com

 

I understand initial admin identity but what is  

Node Identity for 3 nodes i have to update in authorizers.xml file 

 

 

2 ACCEPTED SOLUTIONS

avatar
Explorer

Hi @MattWho one last question 

 

since i have generated certs as per hostnames with proper cn & san..

to configure truststore, do i need to merge the truststore.jks generated for both the hosts or can i use one truststore for both hosts? 

 

sh /opt/nifi-toolkit-1.9.2/bin/tls-toolkit.sh standalone -B  mypasswd -C 'CN=nifiadmin, OU=NIFI' -n 'ip-10-175-12x-xx.abc.com,ip-10-175-12x-xxx.abc.com' --nifiDnPrefix 'CN=' --nifiDnSuffix ', OU=NIFI' -o /tmp/certs_divya/ -K mypasswd -P mypasswd -S mypasswd

 

-rw-------. 1 root root 3437 Feb 3 04:46 CN=nifiadmin_OU=NIFI.p12
-rw-------. 1 root root 29 Feb 3 04:46 CN=nifiadmin_OU=NIFI.password
drwx------. 2 root root 71 Feb 3 04:46 ip-10-175-12x-xxx.abc.com
drwx------. 2 root root 71 Feb 3 04:46 ip-10-175-12x-xxx.abc.com
-rw-------. 1 root root 1200 Feb 3 04:46 nifi-cert.pem
-rw-------. 1 root root 1675 Feb 3 04:46 nifi-key.key

View solution in original post

avatar
Master Mentor

@DivyaKaki 

 

Since all your certificates have been signed by the same CA, the truststore used by all nodes only needs to contain the public cert for that one CA. 

 

Thanks,

Matt

View solution in original post

22 REPLIES 22

avatar
Explorer

@MattWho i also noticed users.xml and authorizations.xml empty

 

cat users.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
<groups/>
<users/>
</tenants>

 

cat authorizations.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizations>
<policies/>
</authorizations>

 

avatar
Master Mentor

@DivyaKaki 

 

Lets take a step back for a moment here....

What is the full version of NiFi you are running currently?

What is set value for following properties in your nifi.properties configuration file:

nifi.security.user.authorizer=file-provider

 

Can you share the contents of your authorizers.xml file?

Looking at what you shared earlier indicates your file-provider is commented out:

<!-- <authorizer>
<identifier>file-provider</identifier>
<class>org.apache.nifi.authorization.FileAuthorizer</class>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Users File">./conf/users.xml</property>
<property name="Initial Admin Identity">CN=NifiAdmin, OU=ApacheNIfi</property>
<property name="Legacy Authorized Users File"></property>

<property name="Node Identity 1">CN=localhost, OU=NIFI</property>
</authorizer>

Note that "<!--" indicates the start of a comment block.  The comment vlock continues until "-->" is found in the XML.

The file-provider should look like this:

<authorizers>
     <authorizer>
         <identifier>file-provider</identifier>
         <class>org.apache.nifi.authorization.FileAuthorizer</class>
         <property name="Authorizations File">./conf/authorizations.xml</property>
         <property name="Users File">./conf/users.xml</property>
         <property name="Initial Admin Identity">CN=NifiAdmin, OU=ApacheNIfi</property>
         <property name="Legacy Authorized Users File"></property>

         <property name="Node Identity 1">CN=localhost, OU=NIFI</property>
     </authorizer>
</authorizers>

If your provider is in fact still commented out, that will explain why your users.xml and authorizations.xml files are not being correctly built on startup.

 

Thank you,

Matt

 

 

avatar
Explorer

@MattWho you are correct, I have uncommented the authorizers file-provider part now, this time generated users.xml and authorizers.xml with content but still same authorization issue 

 

from cert prompt :

Issued to: CN=NifiAdmin,OU=ApacheNIfi

 

authorizers config :

<property name="Initial Admin Identity">CN=NifiAdmin, OU=ApacheNIfi</property>

 

seems I have added space in config file , let me correct it 

avatar
Explorer

@MattWho still got the authorization issue [

No applicable policies could be found. Contact the system administrator.

]here is the cert prompt from browser

 

Issued to: CN=NifiAdmin,OU=ApacheNIfi
Serial number: 01:6F:E9:2D:9F:5C:00:00:00:00
Valid from January 27, 2020, 4:44:38 PM GMT-6 to January 26, 2023, 4:44:38 PM GMT-6
Key Usages: Signing,Non-repudiation,Key Encipherment,Data Encipherment,Key Agreement
Issued by: CN=localhost,OU=NIFI
Stored on: Software Security Devic

 

tried from all nodes from the cluster, same error from all the nodes 

but i have generated client cert 'CN=NifiAdmin, OU=ApacheNIfi_UCSD'

 

sh tls-toolkit.sh standalone -n 'localhost(3)' -C 'CN=NifiAdmin, OU=ApacheNIfi' -o /opt/nifi-toolkit-1.9.2/certs/

 

<authorizer>
<identifier>file-provider</identifier>
<class>org.apache.nifi.authorization.FileAuthorizer</class>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Users File">./conf/users.xml</property>
<property name="Initial Admin Identity">CN=NifiAdmin, OU=ApacheNIfi</property>
<property name="Legacy Authorized Users File"></property>

<property name="Node Identity 1">CN=localhost, OU=NIFI</property>
</authorizer>

 

users.log

 

2020-01-30 01:11:56,526 INFO [NiFi Web Server-36] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: OpenId Connect is not configured.. Returning Conflict response.
2020-01-30 01:11:56,633 INFO [NiFi Web Server-28] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=NifiAdmin, OU=ApacheNIfi) GET https://localhost:9696/nifi-api/flow/current-user (source ip: 10.175.124.142)
2020-01-30 01:11:56,633 INFO [NiFi Web Server-28] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=NifiAdmin, OU=ApacheNIfi
2020-01-30 01:11:56,635 INFO [NiFi Web Server-28] o.a.n.w.a.c.AccessDeniedExceptionMapper identity[CN=NifiAdmin, OU=ApacheNIfi], groups[] does not have permission to access the requested resource. No applicable policies could be found. Returning Forbidden response.

 

cat users.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
<groups/>
<users>
<user identifier="c22273fa-7ed3-38a9-8994-3ed5fea5d23" identity="CN=localhost, OU=NIFI"/>
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b" identity="CN=NifiAdmin, OU=ApacheNIfi"/>
</users>
</tenants>

 

 

i see client cert generated is updated correctly in authorzers.xml but still when I'm trying to log in, getting an authorization error 

No applicable policies could be found. Contact the system administrator.

avatar
Explorer

users.xml and authorizations.xml

I even see <user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86" identity="CN=NifiAdmin, OU=ApacheNIfi"/  is having polices in authorizations.xml file 

 

 

cat authorizations.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizations>
<policies>
<policy identifier="f99bccd1-a30e-3e4a-98a2-dbc708edc67f" resource="/flow" action="R">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
</policy>
<policy identifier="0c88c9dc-6b8c-3146-841a-491a43f4bb5e" resource="/data/process-groups/e5866460-016f-1000-642a-23196f1563d1" action="R">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
<user identifier="c22273fa-7ed3-38a9-8994-3ed5fea5d234"/>
</policy>
<policy identifier="b47dc88e-58b2-34c3-a232-4def8d5cfac9" resource="/data/process-groups/e5866460-016f-1000-642a-23196f1563d1" action="W">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
<user identifier="c22273fa-7ed3-38a9-8994-3ed5fea5d234"/>
</policy>
<policy identifier="6e975e1b-0e66-304e-b749-2d0d83b2b2b7" resource="/process-groups/e5866460-016f-1000-642a-23196f1563d1" action="R">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
</policy>
<policy identifier="ed129295-2501-3536-9280-616f7bbf8a5b" resource="/process-groups/e5866460-016f-1000-642a-23196f1563d1" action="W">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
</policy>
<policy identifier="b8775bd4-704a-34c6-987b-84f2daf7a515" resource="/restricted-components" action="W">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
</policy>
<policy identifier="627410be-1717-35b4-a06f-e9362b89e0b7" resource="/tenants" action="R">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
</policy>
<policy identifier="15e4e0bd-cb28-34fd-8587-f8d15162cba5" resource="/tenants" action="W">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
</policy>
<policy identifier="ff96062a-fa99-36dc-9942-0f6442ae7212" resource="/policies" action="R">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
</policy>
<policy identifier="ad99ea98-3af6-3561-ae27-5bf09e1d969d" resource="/policies" action="W">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
</policy>
<policy identifier="2e1015cb-0fed-3005-8e0d-722311f21a03" resource="/controller" action="R">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
</policy>
<policy identifier="c6322e6c-4cc1-3bcc-91b3-2ed2111674cf" resource="/controller" action="W">
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86"/>
</policy>
<policy identifier="287edf48-da72-359b-8f61-da5d4c45a270" resource="/proxy" action="W">
<user identifier="c22273fa-7ed3-38a9-8994-3ed5fea5d234"/>
</policy>
</policies>
</authorizations>

 


 cat users.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
<groups/>
<users>
<user identifier="c22273fa-7ed3-38a9-8994-3ed5fea5d234" identity="CN=localhost, OU=NIFI"/>
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86" identity="CN=NifiAdmin, OU=ApacheNIfi"/>
</users>
</tenants>

avatar
Explorer

@MattWho i am on nifi 1.9.2 and nifi.property file updated nifi.security.user.authorizer=file-provider

# security properties #
nifi.sensitive.props.key=
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
nifi.sensitive.props.provider=BC
nifi.sensitive.props.additional.keys=

nifi.security.keystore=./conf/keystore.jks
nifi.security.keystoreType=jks
nifi.security.keystorePasswd=xxxx
nifi.security.keyPasswd=xxxx
nifi.security.truststore=./conf/truststore.jks
nifi.security.truststoreType=jks
nifi.security.truststorePasswd=xxxx
nifi.security.needClientAuth=true
#nifi.security.user.authorizer=managed-authorizer

nifi.security.user.authorizer=file-provider
#nifi.security.user.login.identity.provider=ldap-provider

 

the latest error is

An unexpected error has occurred .  HOME

 

javax.net.ssl.SSLPeerUnverifiedException: Hostname ip-10-175-xxx-xxx.abc.com not verified: certificate: sha256/sYgF90RpxFPzPnZJD0jMl0jB/dwS/OgHWxTZ1Ba9TPs= DN: CN=localhost, OU=NIFI subjectAltNames: [localhost]
 
2020-01-30 05:35:03,465 INFO [NiFi Web Server-422441] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response.
2020-01-30 05:35:03,662 INFO [NiFi Web Server-377545] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: OpenId Connect is not configured.. Returning Conflict response.
2020-01-30 05:35:03,766 INFO [NiFi Web Server-422441] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=NifiAdmin, OU=ApacheNIfi) GET https://localhost:9696/nifi-api/flow/current-user (source ip: 10.175.xxx.xxx)
2020-01-30 05:35:03,768 INFO [NiFi Web Server-422441] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=NifiAdmin, OU=ApacheNIfi
 

 

 

avatar
Explorer

@MattWho just want to let you know, i have generated certs using below, belive in this case no need to merge truststore, as it wll be same for all localhost[1-3]

 

 

  sh tls-toolkit.sh standalone -n 'localhost(3)' -C 'CN=NifiAdmin, OU=ApacheNIfi' -o /opt/nifi-toolkit-1.9.2/certs/ssl

 

im still not sure why it is complaining about TLS ssl handshake

An unexpected error has occurred

  • home

javax.net.ssl.SSLPeerUnverifiedException: Hostname ip-10-175-124-250.xxx.com not verified: certificate: sha256/paSWoCMWrMfMuhIjqgLJlyF5clz1RGvEHTDuwhHKeZo= DN: CN=localhost, OU=NIFI subjectAltNames: [localhost]

 

 

authorizers.xml , same for all 3 nodes 

 

<authorizer>
<identifier>file-provider</identifier>
<class>org.apache.nifi.authorization.FileAuthorizer</class>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Users File">./conf/users.xml</property>
<property name="Initial Admin Identity">CN=NifiAdmin, OU=ApacheNIfi</property>
<property name="Legacy Authorized Users File"></property>

<property name="Node Identity 1">CN=localhost, OU=NIFI</property>
</authorizer>

 

users.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
<groups/>
<users>
<user identifier="c22273fa-7ed3-38a9-8994-3ed5fea5d234" identity="CN=localhost, OU=NIFI"/>
<user identifier="5e87a461-6268-359c-97ba-b27d32bc2b86" identity="CN=NifiAdmin, OU=ApacheNIfi"/>
</users>
</tenants>

 

Appreciate if you can help

 

 

 

avatar
Master Mentor

@DivyaKaki 

It is complaining because the hostname is "ip-10-175-124-250.xxx.com" but that hostname does not exist as SAN in your certificate.

 

I would also be concerned that in the nifi.properties files on each of you nodes you are using localhost as your hostnames.  That will not work since that would already resolve to the server itself, so nodes would not be successful at heart beating to one another.

As i mentioned much earlier, the keystore has requirements:
1. keystore can only contain ONE PrivateKeyEntry
2. PrivateKeyEntry must support clientAuth and serverAuth EKU
3. PrivateKeyEntry must contain at least one Subject Alternative Name (SAN) that matches the hostname of the server on which the certificate is being used.  In your case you have "localhost" as a SAN entry, but no SAN entry for "ip-10-175-124-250.xxx.com"

 

This kinda loops back around to the original command i provided that produced keystores not using localhost.


On a side note, looks like after removing the users.xml, you got an expected users.xml created.

Now you may be able to cheat here, but I strongly recommend against it for security reasons (but using localhost is already a bad security idea).  In the nifi.properties file, you could add each of your servers hostnames to the "nifi.web.proxy.host" property (comma separated list with no spaces).

Matt

avatar
Explorer

thanks, @MattWho  for helping me on this, really helped for learning this stuff.

I am now working on using the queries you shared with Harden the security.

 

sh /opt/nifi-toolkit-1.9.2/bin/tls-toolkit.sh standalone -B  mypasswd -C 'CN=nifiadmin, OU=NIFI' -n 'ip-10-175-12x-xx.abc.com,ip-10-175-12x-xxx.abc.com' --nifiDnPrefix 'CN=' --nifiDnSuffix ', OU=NIFI' -o /tmp/certs_divya/ -K mypasswd -P mypasswd -S mypasswd

 

-rw-------. 1 root root 3437 Feb 3 04:46 CN=nifiadmin_OU=NIFI.p12
-rw-------. 1 root root 29 Feb 3 04:46 CN=nifiadmin_OU=NIFI.password
drwx------. 2 root root 71 Feb 3 04:46 ip-10-175-12x-xxx.abc.com
drwx------. 2 root root 71 Feb 3 04:46 ip-10-175-12x-xxx.abc.com
-rw-------. 1 root root 1200 Feb 3 04:46 nifi-cert.pem
-rw-------. 1 root root 1675 Feb 3 04:46 nifi-key.key

 

quick question 

1) validate Keystore of 1 host:

i am seeing two certs in a Keystore as below  but the issuer is localhost whereas CN is a hostname 

is this correct . ?

2)

can I use one truststore from any host and copy to others or do I need to still merge the truststores 

 


Certificate[1]:
Owner: CN=ip-10-175-12x-xxx.abc.com, OU=NIFI
Issuer: CN=localhost, OU=NIFI

 

Certificate[2]:
Owner: CN=localhost, OU=NIFI
Issuer: CN=localhost, OU=NIFI

--------

keytool -list -v -keystore keystore.jks

Enter keystore password:
Keystore type: jks
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: nifi-key
Creation date: Feb 3, 2020
Entry type: PrivateKeyEntry
Certificate chain length: 2
Certificate[1]:
Owner: CN=ip-10-175-12x-xxx.abc.com, OU=NIFI
Issuer: CN=localhost, OU=NIFI
Serial number: 170095f43a400000000
Valid from: Mon Feb 03 04:46:43 UTC 2020 until: Thu Feb 02 04:46:43 UTC 2023
Certificate fingerprints:
MD5: B0:FC:09:D3:A3:40:67:AD:38:EC:30:56:A6:CB:53:89
SHA1: A1:0E:F9:AC:20:B7:9A:AF:D2:C1:B2:DB:1B:80:3F:3C:01:1C:80:3F
SHA256: 8F:18:EC:DD:2D:DC:B9:70:32:07:6B:60:66:7E:21:E5:66:6A:79:FF:65:5E:66:DD:D0:16:F8:C0:8F:87:03:3A
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions:

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: A4 D7 F2 A6 D0 76 CF 42 9E 78 D0 78 20 DF 63 7B .....v.B.x.x .c.
0010: FA E8 F2 BC ....
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
CA:false
PathLen: undefined
]

#3: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
clientAuth
serverAuth
]

#4: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
DigitalSignature
Non_repudiation
Key_Encipherment
Data_Encipherment
Key_Agreement
]

#5: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
DNSName: ip-10-175-124-182.ucsd.edu
]

#6: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 32 1C 2E 86 0C CA 3A E9 1A B8 FD 8E B0 F7 D5 52 2.....:........R
0010: A9 25 EB 74 .%.t
]
]

Certificate[2]:
Owner: CN=localhost, OU=NIFI
Issuer: CN=localhost, OU=NIFI
Serial number: 170095f420d00000000
Valid from: Mon Feb 03 04:46:42 UTC 2020 until: Thu Feb 02 04:46:42 UTC 2023
Certificate fingerprints:
MD5: 8A:13:63:54:E7:E7:E3:E6:FC:16:9E:3B:D0:9D:41:58
SHA1: F5:D0:31:33:5C:AD:9A:39:B5:BF:33:6D:5E:52:A4:F0:5F:99:4C:10
SHA256: 70:1A:ED:41:B7:9C:51:62:B5:26:42:A0:31:9D:D7:04:79:78:FF:9B:89:26:DF:74:24:62:EF:EF:85:4C:E1:E5
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions:

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: A4 D7 F2 A6 D0 76 CF 42 9E 78 D0 78 20 DF 63 7B .....v.B.x.x .c.
0010: FA E8 F2 BC ....
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
CA:true
PathLen:2147483647
]

#3: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
clientAuth
serverAuth
]

#4: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
DigitalSignature
Non_repudiation
Key_Encipherment
Data_Encipherment
Key_Agreement
Key_CertSign
Crl_Sign
]

#5: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: A4 D7 F2 A6 D0 76 CF 42 9E 78 D0 78 20 DF 63 7B .....v.B.x.x .c.
0010: FA E8 F2 BC ....
]
]

 

*******************************************
*******************************************

 

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore keystore.jks -destkeystore keystore.jks -deststoretype pkcs12"

avatar
Master Mentor

@DivyaKaki 

 

If you are trying to harden your security, using localhost as the CN for your certificate authority is not good security practice.

Your keystore PrivateKeyEntry structure is good.  You have 1 entry.  That one entry includes two certificates.  The first is for the server on which this certificate is being used.  The second is the public certificate for the CA that signed certificate 1 in this chain.  Certificate 1 has the proper EKUs and a good SAN.

Please take a moment to accept the answers that helped solved your query.

Matt