Minifi C++ 0.4.0 secure communication

[#1]

Has anyone got Minifi C++ v0 4 0 working over a secure connection to a Nifi cluster. My certs seem to be fine as I can log into UI of the secured cluster through the browser using a cert I created for a nifiadmin user. I've got minifi (collecting Squid logs) working unsecured but every time I try to get it connecting securely it won't authenticate to be able to retreive s2s settings. The logs are:

[2018-03-10 19:13:58.149] [org::apache::nifi::minifi::controllers::SSLContextService] [info] not good [2018-03-10 19:13:58.149] [org::apache::nifi::minifi::controllers::SSLContextService] [warning] still not good [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 2ef87a2c-2497-11e8-81ff-be2999981f84 from Get_access.log to relationship success [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::processors::TailFile] [info] TailFile access.log for 2481982 bytes [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'Store State' of flow file '2ef87a2c-2497-11e8-81ff-be2999981f84' with value 'Do not store state' [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'host_name' of flow file '2ef87a2c-2497-11e8-81ff-be2999981f84' with value 'gs1' [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'tenant' of flow file '2ef87a2c-2497-11e8-81ff-be2999981f84' with value ''TheBerties'' [2018-03-10 19:13:58.230] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 2ef87a2c-2497-11e8-81ff-be2999981f84 from UpdateAttribute to relationship success [2018-03-10 19:13:58.426] [org::apache::nifi::minifi::utils::HTTPClient] [error] curl_easy_perform() failed Peer certificate cannot be authenticated with given CA certificates [2018-03-10 19:13:58.426] [org::apache::nifi::minifi::RemoteProcessorGroupPort] [error] ProcessGroup::refreshRemoteSite2SiteInfo -- curl_easy_perform() failed [2018-03-10 19:13:58.426] [org::apache::nifi::minifi::c2::C2Agent] [info] Class is RESTSender [2018-03-10 19:13:58.428] [org::apache::nifi::minifi::io::Socket] [error] Could not bind to socket [2018-03-10 19:13:58.429] [org::apache::nifi::minifi::FlowController] [info] Started Flow Controller [2018-03-10 19:13:58.429] [main] [info] MiNiFi started [2018-03-10 19:13:58.444] [org::apache::nifi::minifi::utils::HTTPClient] [error] curl_easy_perform() failed Peer certificate cannot be authenticated with given CA certificates [2018-03-10 19:13:58.444] [org::apache::nifi::minifi::RemoteProcessorGroupPort] [error] ProcessGroup::refreshRemoteSite2SiteInfo -- curl_easy_perform() failed [2018-03-10 19:13:58.444] [org::apache::nifi::minifi::RemoteProcessorGroupPort] [info] no protocol, yielding

I can't find much in the way of documentation for secure minifi cpp operations. I'm using the Centos 7 version on Hortonworks repository. Has anyone been successful and are you able to suggest what I'm doing wrong. I haven't reverted back to compiling from github yet but guess that is the next on the list.

Many thanks in advance.

Tom

[#2]

@mparisi, that is an awesome response, which I will try to action this evening when I'm able to get close enough to my environment to play around. The guilty minifi.properties file is the one contained in the Hortonworks bin for Centos 7 at http://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.1.1.0/tars/nifi-minifi-cpp/nifi-minif...

Many thanks and I'll get back to you (hopefully with a positive resolution), Tom

[#3]

And to answer your other question @mparisi, yes I'm using Nifi CA and generating certs with tls-toolkit.

[#4]

Building the latest source is usually good for odd problems.

But this is pretty standard and should work. Engineering will post some more information.

Have you tested your certificates?

https://www.cyberciti.biz/faq/test-ssl-certificates-diagnosis-ssl-certificate/

Was your MiniFi C++ built with SSL enabled?

type curl -V

[#5]

Thanks for your response and suggestions @Timothy Spann. I've tested the certs and they seem to be OK. I'm getting a self-signed response of 19 and when I have copied the CA cert into /etc/pki/ca-trust/source/anchors/ it responds 0(OK). Still not joy in getting things connected though. I'm about to start a fresh build from source in the hope that will change things.

[#6]

Sounds good. Someone from engineering hopefully will help you run this down. I am trying to think what other logs might be useful to post to spot something odd.

[#7]

@Tom Burton

Sorry for the delay. I responded but duplicated my response and upon deleting it must have deleted both. Yesterday I submitted https://github.com/apache/nifi-minifi-cpp/pull/276

I tested this on CENTOS 7 . I first was able to get it working by adding my nifi-cert to the ca-cert bundle in /etc/ssl ; however, I recognize that not everyone wants to do this, so I created the above pull request for review. This approach may not be correct in all cases, so I may update the PR to give users an option to use the system defined CA bundle ( my preference in certain deployments); however, if you have the chance to test it you may be able to get around not having to disable peer verification and updating the system's CA bundle ( that is curl's default CAFile/CAPath).

[#8]

In addition to the above response, on my prior response I mentioned that I was getting a domain error after adding my cert to the CA bundle and using the above fix but re-issues my cert so that the domain matched the certificate.

[#9]

A quick update @Timothy Spann and @mparisi. I've built from the current master on GitHub and am getting largely the same problems (just slightly different phrasing), which has led me to think again about what might be different about my environment. My standard centos VM is built with a NIST 800-171 security profile applied, which is where my thinking is at the moment. I'll have to look at that profile now to see what clamps may be causing the problem and gradually unwind them. Any ideas you may have would be great. It's going to be important to identify the blocker (if this is the cause) because many of my clients will have various security policies applied to the servers.

[#10]

@Tom Burton Thanks for the info. I'll investigate with that profile and make any changes that are needed in the PR, above. Thanks!

[#11]

That may lock down more things. Maybe you need a stronger cert instead of defaults.