Silver ⭐️⭐️

Solved

nifi security configuration

6 replies
7520 views
Translate All

Hello

I've been setting up my secured nifi cluster with the help of the nifi-tls-toolkit in standalone mode. which outputs the nifi.properties file along with keystores and the certificate.

I tried to use the tls-toolkit in client/server mode, which outputs the same keystores and certificate. but instead of outputting the nifi.properties file it outputs a config.json containing the keystore passwords,types etc. in a json file. In the nifi docs (https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#client-server) nothing is really mentioned about the config.json file, accept configuration of the tls-toolkit where to output it (client) /read it from (server). and if i choose to use this method (client/server) the only valid solution for configuration of the nifi security properties is to copy the json fields into the nifi.properties of the node the certificate was created for (manually or via script).

The main question i'm interested in is what is this config.json file?, is it documented anywhere?, can i count that the structure of this file remaining the same in the future?. or better, can i configure nifi security properties to be read from the config.json file instead of them being read from the nifi.properties file?.

Apache NiFi

MattWhoAnswer

Titanium ⭐️⭐️⭐️

@Nimrod Avni

-

The Standalone option is not ideal for setting up a NiFi cluster. Since the certificates generated are not signed by a Certificate Authority, the truststore will need to contain a trustedCertEntry for each certificate created. Adding additional nodes to a cluster would require going back and modifying the truststore on every node in the cluster.

-

The Client/Server mode allows you to standup a Certificate Authority (Server mode) that will be used to sign all the client certificate created (one for each NiFi node). When you stand up the Server a config.json is generated which can be used as input to the client mode operation. Because of this it is common that each of the client certificates are also generated from same server where the CA (server) was created/started. The client mode simply outputs a config.json file for each client certificate which simply provides the information needed to setup the relevant nifi.properties properties on each of your NiFi nodes.

-

It is safe to say that the structure will remain unchanged with a major NiFi release version.

An external script could be used to update a nodes nifi.properties file from the output generated in the client mode config.json file.

HDF for example already does this. If you choose to utilize the NiFi CA in HDF, it will take care of obtaining the client certificates and updating the nifi.properties on each node. This allows new client certificate to be generated on demand for each node.

There is no option to configure NiFi to read these security parameters from the client mode generated config.json file.

-

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.

Eran

Gold ⭐️

The tls-toolkit.sh supports --configJsonIn in the same guide that you reference

Translate

nimrodavni7Author

Silver ⭐️⭐️

If i understand correctly, I can supply the server with configJson file with the keystore passwords/types etc and whenever i request a new certificate with the client, the server will use the properties from the configJson file.

My question is what is exactly this configJson file?, is it a common format that Certificate authorities store their configuration, or just some format decided by the developers of the nifi toolkit?

Translate

MattWho

Titanium ⭐️⭐️⭐️

@Nimrod Avni

The config.json generated as output when you stood up your NIFi CA (server) is there to simplify the execution of the client mode so that you do not have to manually pass all the server info to the client mode input. This was just a choice made by the development team to generate this file rather then just expect user to remember what they entered when the stood up the Server. You can delete this file if you want to as long as you have stored or can remember the pertinent information yourself for running the tls-toolkit client mode later.
as far as client mode goes, the generated config.json is also just there to provide you the pertinent information about the client keystore that was created this is all information you should already know (unless you did not provide a password and toolkit auto-generated one for you which they you would need to get form the output config.json file.)

-

Thanks,

Matt

Translate

MattWhoAnswer

Titanium ⭐️⭐️⭐️

@Nimrod Avni

-

The Standalone option is not ideal for setting up a NiFi cluster. Since the certificates generated are not signed by a Certificate Authority, the truststore will need to contain a trustedCertEntry for each certificate created. Adding additional nodes to a cluster would require going back and modifying the truststore on every node in the cluster.

-

The Client/Server mode allows you to standup a Certificate Authority (Server mode) that will be used to sign all the client certificate created (one for each NiFi node). When you stand up the Server a config.json is generated which can be used as input to the client mode operation. Because of this it is common that each of the client certificates are also generated from same server where the CA (server) was created/started. The client mode simply outputs a config.json file for each client certificate which simply provides the information needed to setup the relevant nifi.properties properties on each of your NiFi nodes.

-

It is safe to say that the structure will remain unchanged with a major NiFi release version.

An external script could be used to update a nodes nifi.properties file from the output generated in the client mode config.json file.

HDF for example already does this. If you choose to utilize the NiFi CA in HDF, it will take care of obtaining the client certificates and updating the nifi.properties on each node. This allows new client certificate to be generated on demand for each node.

There is no option to configure NiFi to read these security parameters from the client mode generated config.json file.

-

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.

Translate

nimrodavni7Author

Silver ⭐️⭐️

Thank you for your answer, it is very helpful, but does bring up some follow up questions.

If the client/server mode is designed for the setup of secured clusters, why should the certificate be created in the same server where the CA (tls-toolkit server mode) is running and not in the server that i plan to run my node. Further more, it is very strange that the output of the tls-toolkit (client) requesting for a certificate comes out as a config.json file (containing all the properties that should be configured in the nifi.properties) and it is not possible to configure nifi to read those properties from the config.json file. it seems like its not really built for automatic scaling of nodes, and most times will require the intervention of a person or a 3rd party script to automate that scaling.

If all the certificates should created on the same server when running in a client/server architecture, isn't it more beneficial to have one server dedicated for creating certificates, running in standalone mode, so all the certificates are signed by the same CA, while outputting the nifi.propeties file ready to be used for the deployment of the node?

Thank You once again,

Nimrod

Translate

MattWho

Titanium ⭐️⭐️⭐️

You can run the tls-toolkit in client mode directly from any node, but you will either need to provide the CA server info or copy CA config.json to each node manually. I was not trying to imply that you must execute client mode form same server where CA server was installed.
-
The NiFi CA was not built with the intent for use in a production environment. It was built as a tool that allows users to easily and quickly setup secured NiFi instances/clusters for development and testing purposes. For production environments a corporately/privately managed CA should be used.

-

There should only ever be one NiFi CA installed and being used to sign all certificates. I apologize if what i wrote was confusing and led you to believe multiple NiFi CAs were needed or should be used.

-

Feel free to open an Apache NiFi Jira to add the ability to update an existing or output a new nifi.properties file when client mode is used. I don't see that as a bad request at all.

-

Thank you,

Matt

Translate

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.