Apache NiFi: How to configure a Remote Process Group with enabled Certificate Authentication?

[#1]

Hi,

I want to configure a Remote Process Group in my NiFi Flow, but I get a SSLPeerUnverifiedException. Do I have to configure a certificate for the RPG? How can I import the certificate for the RPG?

Thank you in advance!

Kind regards,

Jan

[#2 · Accepted solution]

@Jan

This error is telling you the hostname in the URL header does not match the owner or any Subject Alternative Names (SAN) in the server certificate being presented by the target server.

For example: You configure an RPG with a target NiFi URL of "https://nifiserver:9443/nifi". During the TLS handshake the server identifies itself based on the contents of its keystore as "nifiserver01.corp.com". The above error is thrown because it appears some other server is trying to pretend to be intended destination server "nifiserver".

NiFi's Site-to-SIte (S2S) capability uses the keystore and truststore configured in the nifi.properties file of both NiFi instances (Client side running RPG and server side running input/output ports) for the TLS handshake.

You will want to perform a verbose listing of the keystore file on your target Nifi and make sure the following are true:

1. The keystore contains only a single PrivateKeyEntry

2. The owner of that single PrivateKeyEntry has a CN value or a SAN entry that matches the hostname of that server.

Then make sure that you are using either the owner CN or one of the SAN entries in the URL used in your client side RPG.

Thanks,

Matt

[#3]

@Jan

Tip: add comments to existing answers unless you are really starting a new answer thread.

From a security standpoint, using "localhost" in a certificate is not a good idea.

As far as SAN entries go, they must be added during the certificate creation process.

I am not clear on why you would build a dataflow that will be used on a MiNiFi instance to talk back to itself (MiNiFi)? Or are you trying to create a dataflow to convert into a yml file which will include an RPG that is pointing back to your main NiFi instance.

If the latter is the case, you will need to have separate keystore and truststore for each NiFi instance (MiNiFi and NiFi). As long as all certificates are signed by the same CA, a single truststore that contains a trustedCertEntry for that shared CA can be used on both MiNiFi and NiFi installs. If both certificates are self signed, you will need to make sure the public cert is loaded in to each others truststores or the TLS handshake will fail.

If you are just creating keystores and truststores to play around with, you could use something like tinycerts to generate them for you. https://www.tinycert.org/

Thank you,

Matt

If you found this answer addressed your original question, please take a moment to click "accpet" below the answer.

[#4]

Thank you for your answer.

I would like to follow the tutorial: https://de.hortonworks.com/tutorial/analyze-iot-weather-station-data-via-connected-data-architecture..., but I am struggeling with step 3.1. So, yes I want to "create a dataflow to convert it into a yml file which will include a RPG that is pointing back to your main NiFi instance". That means, that the main NiFi instance is talking to itself and during that configuration the error occurs. But as long as I cannot configure the RPG, I cannot connect the "ExecuteProcess" to the RPG. That is the reason why I need to configure the RPG to talk to the main NiFi instance.

King regards,

Jan

[#5]

@Jan

What process did you follow to create the keystore and truststore you are using now?

You will need to create a new keystore and truststore.

Thanks,

Matt

[#6]

@Matt Clarke

I used the command: ./bin/tls-toolkit.sh standalone -n '<ip address of the nifi server>' -C 'CN=<username>, OU=NIFI' -o './target' and followed the tutorial (but only one node instead of 2): https://bryanbende.com/development/2016/08/17/apache-nifi-1-0-0-authorization-and-multi-tenancy

Do I need another command/argument to generate the keystore and truststore?

Kind regards,

Jan

[#7]

@Jan

Why only one node? Are you installing both NiFi and MiNiFi on the same server?

I would execute the 'hostname -f" command on both the server where NiFi is installed and the server wheer MiNiFi is/will be installed.

Then use those full hostnames in the tls-tookit.sh script to create two keystores for use on these two instances

./tls-toolkit.sh standalone -n '<nifi.hostname>,<minifi.hostname>' -C 'CN=<nifiuser>, OU=NIFI' -P <truststore passsword> -S <keystorepassword> 

You will end up with a directory for each server being created that contains the needed TLS configuration info/files.

If you want to include SAN to your new certificates, you will need to create each one at a time:

./tls-toolkit.sh standalone -n '<nifi.hostname>' -C 'CN=<nifiuser>, OU=NIFI' -P <truststore passsword> -S <keystorepassword> --subjectAlternativeNames '<nifi-ip-address>,<etc>'


./tls-toolkit.sh standalone -n '<minifi.hostname>' -P <truststore passsword> -S <keystorepassword> --subjectAlternativeNames '<minifi-ip-address>,<etc>'

You should not need to any SAN entry if you generate your keystores using the actual hostnames assigned to your servers.

Also make sure that the following properties have been set to the appropariate hostnames as well in the nifi.properties file:

nifi.remote.input.host=
nifi.web.https.host=

Once your NiFi server is using the new keystore and truststore files, you can use the nifi server hostname in the RPG.

Thanks,

Matt

[#8]

Hi @Matt Clark,

thank you for your answer! Actually I am trying to build a rpg on the server itself, so it tries to connect to himself (I want to configure the dataflow for MiNiFi).

I verified point 1. The keystore has only one entry. How do I configure the SAN? The owner CN is configured as 'localhost'. That might be the problem.

Kind regards,

Jan

[#9]

thank you @Matt Clarke! Now it seems to be working, but the RPG throws another exception:

Do I need to create a specific user and set the policy "retrieve data via site-to-site" in the Input Port of the nifi flow?

[#10]

@Jan

Can we keep each post in community.hortonworks.com to a single question? The above is no longer an TLS handshake issue. It is an authorization issue. Please start a new question for this topic.

Accepting an answer for this question will close this thread.

Thank you,

Matt