Minifi C++ 0.4.0 secure communication

[#1]

Has anyone got Minifi C++ v0 4 0 working over a secure connection to a Nifi cluster. My certs seem to be fine as I can log into UI of the secured cluster through the browser using a cert I created for a nifiadmin user. I've got minifi (collecting Squid logs) working unsecured but every time I try to get it connecting securely it won't authenticate to be able to retreive s2s settings. The logs are:

[2018-03-10 19:13:58.149] [org::apache::nifi::minifi::controllers::SSLContextService] [info] not good [2018-03-10 19:13:58.149] [org::apache::nifi::minifi::controllers::SSLContextService] [warning] still not good [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 2ef87a2c-2497-11e8-81ff-be2999981f84 from Get_access.log to relationship success [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::processors::TailFile] [info] TailFile access.log for 2481982 bytes [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'Store State' of flow file '2ef87a2c-2497-11e8-81ff-be2999981f84' with value 'Do not store state' [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'host_name' of flow file '2ef87a2c-2497-11e8-81ff-be2999981f84' with value 'gs1' [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'tenant' of flow file '2ef87a2c-2497-11e8-81ff-be2999981f84' with value ''TheBerties'' [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 2ef87a2c-2497-11e8-81ff-be2999981f84 from UpdateAttribute to relationship success [2018-03-10 19:13:58.426] [org::apache::nifi::minifi::utils::HTTPClient] [error] curl_easy_perform() failed Peer certificate cannot be authenticated with given CA certificates [2018-03-10 19:13:58.426] [org::apache::nifi::minifi::RemoteProcessorGroupPort] [error] ProcessGroup::refreshRemoteSite2SiteInfo -- curl_easy_perform() failed [2018-03-10 19:13:58.426] [org::apache::nifi::minifi::c2::C2Agent] [info] Class is RESTSender [2018-03-10 19:13:58.428] [org::apache::nifi::minifi::io::Socket] [error] Could not bind to socket [2018-03-10 19:13:58.429] [org::apache::nifi::minifi::FlowController] [info] Started Flow Controller [2018-03-10 19:13:58.429] [main] [info] MiNiFi started [2018-03-10 19:13:58.444] [org::apache::nifi::minifi::utils::HTTPClient] [error] curl_easy_perform() failed Peer certificate cannot be authenticated with given CA certificates [2018-03-10 19:13:58.444] [org::apache::nifi::minifi::RemoteProcessorGroupPort] [error] ProcessGroup::refreshRemoteSite2SiteInfo -- curl_easy_perform() failed [2018-03-10 19:13:58.444] [org::apache::nifi::minifi::RemoteProcessorGroupPort] [info] no protocol, yielding

I can't find much in the way of documentation for secure minifi cpp operations. I'm using the Centos 7 version on Hortonworks repository. Has anyone been successful and are you able to suggest what I'm doing wrong. I haven't reverted back to compiling from github yet but guess that is the next on the list.

Many thanks in advance.

Tom

[#2]

Thanks @mparisi

How do I join apache hipchat? It only gave me the option of entering my email which it didn't recognise, and there didn't appear to be an option to register.

Things seem to be moving forward but, and also a little backwards. I've change my build sequence to be:

bootstrap -> make -> make packagemissing out the cmake step that is described in the README because it noticed bootstrap does this itself). This may have contributed to the problem.

I can now get a connection up and running, including with a build from master rather than PR285. However, it never gets flow controller running, and halts after a final message of "Class is RESTsender". What I can't understand is that there are no errors in the logs at all, but after starting minifi the status reports that it is not running. Sorry if I'm becoming an irritant. The logs are as follows (I've removed the dynamic property warnings to save space):

[root@gs1 nifi-minifi-cpp-0.4.0]# bin/minifi.sh start
PID 2056 is stale, removing pid file at /root/nifi-minifi-cpp-0.4.0/bin/.minifi.pid
Starting MiNiFi with PID 2237 and pid file /root/nifi-minifi-cpp-0.4.0/bin/.minifi.pid
[2018-03-infol 15:12:59.328] [main] [info] Using MINIFI_HOME=/root/nifi-minifi-cpp-0.4.0 from environment.
[2018-03-infol 15:12:59.328] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /root/nifi-minifi-cpp-0.4.0/conf/minifi-log.properties
setting default dir to /root/nifi-minifi-cpp-0.4.0/content_repository
[root@gs1 nifi-minifi-cpp-0.4.0]# cat logs/*
[2018-03-21 15:12:59.329] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /root/nifi-minifi-cpp-0.4.0/conf/minifi-uid.properties
[2018-03-21 15:12:59.329] [main] [info] MINIFI_HOME=/root/nifi-minifi-cpp-0.4.0
[2018-03-21 15:12:59.329] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /root/nifi-minifi-cpp-0.4.0/conf/minifi.properties
[2018-03-21 15:12:59.346] [org::apache::nifi::minifi::FlowController] [info] FlowController NiFi Configuration file /root/nifi-minifi-cpp-0.4.0/conf/config.yml
[2018-03-21 15:12:59.346] [main] [info] Loading FlowController
[2018-03-21 15:12:59.346] [org::apache::nifi::minifi::FlowController] [info] Load Flow Controller from file /root/nifi-minifi-cpp-0.4.0/conf/config.yml
...
[2018-03-21 15:12:59.349] [org::apache::nifi::minifi::FlowController] [info] Loaded root processor Group
[2018-03-21 15:12:59.349] [org::apache::nifi::minifi::FlowController] [info] Initializing timers
[2018-03-21 15:12:59.349] [org::apache::nifi::minifi::FlowController] [info] Loaded controller service provider
[2018-03-21 15:12:59.349] [org::apache::nifi::minifi::FlowController] [info] Loaded flow repository
[2018-03-21 15:12:59.349] [org::apache::nifi::minifi::FlowController] [info] Starting Flow Controller
[2018-03-21 15:12:59.349] [org::apache::nifi::minifi::core::controller::StandardControllerServiceProvider] [info] Enabling % controller services
[2018-03-21 15:12:59.349] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] UpdateAttribute registering 3 keys
[2018-03-21 15:12:59.349] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] UpdateAttribute registered attribute 'Store State'
[2018-03-21 15:12:59.349] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] UpdateAttribute registered attribute 'host_name'
[2018-03-21 15:12:59.349] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] UpdateAttribute registered attribute 'tenant'
[2018-03-21 15:12:59.358] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 5766057a-2d1a-11e8-8703-be2999981f84 from Get_access.log to relationship success
[2018-03-21 15:12:59.358] [org::apache::nifi::minifi::processors::TailFile] [info] TailFile access.log for 2657416 bytes
[2018-03-21 15:12:59.358] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'Store State' of flow file '5766057a-2d1a-11e8-8703-be2999981f84' with value 'Do not store state'
[2018-03-21 15:12:59.358] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'host_name' of flow file '5766057a-2d1a-11e8-8703-be2999981f84' with value 'gs1'
[2018-03-21 15:12:59.358] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'tenant' of flow file '5766057a-2d1a-11e8-8703-be2999981f84' with value ''TheBerties''
[2018-03-21 15:12:59.358] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 5766057a-2d1a-11e8-8703-be2999981f84 from UpdateAttribute to relationship success
[2018-03-21 15:12:59.594] [org::apache::nifi::minifi::RemoteProcessorGroupPort] [info] Have 1 peers
[2018-03-21 15:12:59.594] [org::apache::nifi::minifi::c2::C2Agent] [info] Class is RESTSender

[root@gs1 nifi-minifi-cpp-0.4.0]# bin/minifi.sh status
Program is not currently running but stale pid file (/root/nifi-minifi-cpp-0.4.0/bin/.minifi.pid) exists.

Any guidance would be appreciated. I feel tantalisingly close, but not close enough.

Tom

[#3]

So I have finally got a working solution on this, though it may not be ideal. It also seems to be rather simplistic. Curl, when compiled with NSS, doesn't seem to like encrypted pem files for client certificates/keys. I can make an SSL connection using openssl without any problems using the command (it asks for my password the password for cert.pem and establishes a connection correctly):

openssl s_client -connect host:port -key cert.pem -cert cert.pem -CAfile ca.pem

But if I try to do the same with curl using the following command:

curl -v --cacert ./nifi-cert.pem --cert ./cert.pem:password --key ./cert.pem:password host:port

It consistently fails with the error:

* unable to load client cert: -8018 (SEC_ERROR_UNKNOWN_PKCS11_ERROR)
* NSS error -8018 (SEC_ERROR_UNKNOWN_PKCS11_ERROR)
* Unknown PKCS #11 error.

I've tried forcing openssl to use des3 when converting the pkcs12 file generated by nifi-tools to pem. I've tried playing around with different password strengths. None of this works. If on the other hand, I force openssl to not encrypt the pem certificate, using -nodes it works fine.

Not ideal, because it is not exactly good practice storing keys in clear. On the other hand, the password being used by minifi will be stored in clear so not much more downside doing it this way. I'm sure I can do it another way by storing the certificate in the NSS db (though I did play around with this and there was no easy solution), but I'm keen to minimise the actions required to deploy it.

In the process I also installed nss-devel and nss-pkcs11-devel (both versions 3.28.4) but I don't know whether this had a positive of negative effect. Over the next few days I'll remove them and re-compile to see whether they are dependencies when building for Centos/RHEL 7.

Many thanks to @mparisi and @Timothy Spann for their support and patience on this issue. If anyone has a better way of solving this I'm all ears.

Tom

[#4]

@Tom Burton Sorry for the delay. Using NSS will require some changes. Use a PKCS12 instead of PEM cert types with an extension of p12. This should alleviate some issues. In that case you should still be able to have a key password. This is a restriction in some of the CURL code.

Tom thank you for your patience. I recognize that as a user worrying about NSS vs CURL is something you don't want to concern yourself with so we're exploring two options and will provide both as an option to users:

1) We'd like to create system packages. In this case an RPM that will include all that is necessary to run with curl w/ OpenSSL

2) Package an uber binary with statically linked OpenSSL. This will result in a larger binary but include the necessary dependencies.

Either option would help alleviate your pain points because you won't be having to worry about bootstrapping clients. With CENTOS 7 you will need to manually download curl and build it with OpenSSL. That is not ideal, hence why I think we'll push to get options one and two out the door as soon as possible. Option two is likely more forthcoming as the developer has made great strides, so when that is available we can avoid having to use NSS because the system doesn't come with curl w/ OpenSSL.

If you are willing to join in another test I hope we make things a little easier as I'm sure worrying about P12 vs PEM for the variants of CURL is an unnecessary burden. Thanks again for helping so much with CENTOS 7. Your efforts will definitely be the key that makes that distribution easier to deploy.

[#5]

Thanks @mparisi, and I'm more than happy to test. Your suggestions sound great, because it's rather err on a larger binary and lower client dependencies. If I need a particularly slim build for an alliance I can always build it myself. I did try with pkcs certs (easier because that is what nifi toolkit produces) but had problems with that too. I'm not that bothered clamping access to an unencrypted, though it will make rotating keys a bit more of a challenge.