Thanks @mparisi, and I'm more than happy to test. Your suggestions sound great, because it's rather err on a larger binary and lower client dependencies. If I need a particularly slim build for an alliance I can always build it myself. I did try with pkcs certs (easier because that is what nifi toolkit produces) but had problems with that too. I'm not that bothered clamping access to an unencrypted, though it will make rotating keys a bit more of a challenge. ... View more

So I have finally got a working solution on this, though it may not be ideal. It also seems to be rather simplistic. Curl, when compiled with NSS, doesn't seem to like encrypted pem files for client certificates/keys. I can make an SSL connection using openssl without any problems using the command (it asks for my password the password for cert.pem and establishes a connection correctly): openssl s_client -connect host:port -key cert.pem -cert cert.pem -CAfile ca.pem But if I try to do the same with curl using the following command: curl -v --cacert ./nifi-cert.pem --cert ./cert.pem:password --key ./cert.pem:password host:port It consistently fails with the error: * unable to load client cert: -8018 (SEC_ERROR_UNKNOWN_PKCS11_ERROR) * NSS error -8018 (SEC_ERROR_UNKNOWN_PKCS11_ERROR) * Unknown PKCS #11 error. I've tried forcing openssl to use des3 when converting the pkcs12 file generated by nifi-tools to pem. I've tried playing around with different password strengths. None of this works. If on the other hand, I force openssl to not encrypt the pem certificate, using -nodes it works fine. Not ideal, because it is not exactly good practice storing keys in clear. On the other hand, the password being used by minifi will be stored in clear so not much more downside doing it this way. I'm sure I can do it another way by storing the certificate in the NSS db (though I did play around with this and there was no easy solution), but I'm keen to minimise the actions required to deploy it. In the process I also installed nss-devel and nss-pkcs11-devel (both versions 3.28.4) but I don't know whether this had a positive of negative effect. Over the next few days I'll remove them and re-compile to see whether they are dependencies when building for Centos/RHEL 7. Many thanks to @mparisi and @Timothy Spann for their support and patience on this issue. If anyone has a better way of solving this I'm all ears. Tom ... View more

Thanks @mparisi How do I join apache hipchat? It only gave me the option of entering my email which it didn't recognise, and there didn't appear to be an option to register. Things seem to be moving forward but, and also a little backwards. I've change my build sequence to be: bootstrap -> make -> make packagemissing out the cmake step that is described in the README because it noticed bootstrap does this itself). This may have contributed to the problem. I can now get a connection up and running, including with a build from master rather than PR285. However, it never gets flow controller running, and halts after a final message of "Class is RESTsender". What I can't understand is that there are no errors in the logs at all, but after starting minifi the status reports that it is not running. Sorry if I'm becoming an irritant. The logs are as follows (I've removed the dynamic property warnings to save space): [root@gs1 nifi-minifi-cpp-0.4.0]# bin/minifi.sh start PID 2056 is stale, removing pid file at /root/nifi-minifi-cpp-0.4.0/bin/.minifi.pid Starting MiNiFi with PID 2237 and pid file /root/nifi-minifi-cpp-0.4.0/bin/.minifi.pid [2018-03-infol 15:12:59.328] [main] [info] Using MINIFI_HOME=/root/nifi-minifi-cpp-0.4.0 from environment. [2018-03-infol 15:12:59.328] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /root/nifi-minifi-cpp-0.4.0/conf/minifi-log.properties setting default dir to /root/nifi-minifi-cpp-0.4.0/content_repository [root@gs1 nifi-minifi-cpp-0.4.0]# cat logs/* [2018-03-21 15:12:59.329] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /root/nifi-minifi-cpp-0.4.0/conf/minifi-uid.properties [2018-03-21 15:12:59.329] [main] [info] MINIFI_HOME=/root/nifi-minifi-cpp-0.4.0 [2018-03-21 15:12:59.329] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /root/nifi-minifi-cpp-0.4.0/conf/minifi.properties [2018-03-21 15:12:59.346] [org::apache::nifi::minifi::FlowController] [info] FlowController NiFi Configuration file /root/nifi-minifi-cpp-0.4.0/conf/config.yml [2018-03-21 15:12:59.346] [main] [info] Loading FlowController [2018-03-21 15:12:59.346] [org::apache::nifi::minifi::FlowController] [info] Load Flow Controller from file /root/nifi-minifi-cpp-0.4.0/conf/config.yml ... [2018-03-21 15:12:59.349] [org::apache::nifi::minifi::FlowController] [info] Loaded root processor Group [2018-03-21 15:12:59.349] [org::apache::nifi::minifi::FlowController] [info] Initializing timers [2018-03-21 15:12:59.349] [org::apache::nifi::minifi::FlowController] [info] Loaded controller service provider [2018-03-21 15:12:59.349] [org::apache::nifi::minifi::FlowController] [info] Loaded flow repository [2018-03-21 15:12:59.349] [org::apache::nifi::minifi::FlowController] [info] Starting Flow Controller [2018-03-21 15:12:59.349] [org::apache::nifi::minifi::core::controller::StandardControllerServiceProvider] [info] Enabling % controller services [2018-03-21 15:12:59.349] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] UpdateAttribute registering 3 keys [2018-03-21 15:12:59.349] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] UpdateAttribute registered attribute 'Store State' [2018-03-21 15:12:59.349] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] UpdateAttribute registered attribute 'host_name' [2018-03-21 15:12:59.349] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] UpdateAttribute registered attribute 'tenant' [2018-03-21 15:12:59.358] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 5766057a-2d1a-11e8-8703-be2999981f84 from Get_access.log to relationship success [2018-03-21 15:12:59.358] [org::apache::nifi::minifi::processors::TailFile] [info] TailFile access.log for 2657416 bytes [2018-03-21 15:12:59.358] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'Store State' of flow file '5766057a-2d1a-11e8-8703-be2999981f84' with value 'Do not store state' [2018-03-21 15:12:59.358] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'host_name' of flow file '5766057a-2d1a-11e8-8703-be2999981f84' with value 'gs1' [2018-03-21 15:12:59.358] [org::apache::nifi::minifi::processors::UpdateAttribute] [info] Set attribute 'tenant' of flow file '5766057a-2d1a-11e8-8703-be2999981f84' with value ''TheBerties'' [2018-03-21 15:12:59.358] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 5766057a-2d1a-11e8-8703-be2999981f84 from UpdateAttribute to relationship success [2018-03-21 15:12:59.594] [org::apache::nifi::minifi::RemoteProcessorGroupPort] [info] Have 1 peers [2018-03-21 15:12:59.594] [org::apache::nifi::minifi::c2::C2Agent] [info] Class is RESTSender [root@gs1 nifi-minifi-cpp-0.4.0]# bin/minifi.sh status Program is not currently running but stale pid file (/root/nifi-minifi-cpp-0.4.0/bin/.minifi.pid) exists. Any guidance would be appreciated. I feel tantalisingly close, but not close enough. Tom ... View more

Hello @mparisi. I can't seem to reply to your last message so have started a fresh thread. Thank you for creating Pull 285 for this. I've built from it, and apart from having to manually create the conf directory with its contents the built seemed to compete successfully. It identified that libcurl-nss was isn't rather than openssl, and built to work with that. However, I'm still failing to authenticate. Clearly I'm doing something stupid now. Should I be changing the config in minifi.properties, config.yml or doing anything else differently to make it work with nss. I'll go through everything again to make sure I haven't missed anything out but would welcome an guidance you can provide. Thanks for all your help, Tom ... View more

That works for me @mparisi. Let me know if there is anything I can do to test it. Many thanks, Tom ... View more

Thanks @mparisi, much appreciated and I look forward to the updated bootstrap. Apologies for what is probably a stupid question, but if minifi is built under this bootstrap with libcurl-openssl will this work regardless of the version of curl that is installed on the system that it is running on? ... View more

I've been doing some further investigation @mparisi and I think your observation about the curl version may be at the root. I've tried running it on a vanila Centos 7 VM with no security profile and still have the same problem. I've also tried establishing a session with curl from the command line, and it consistently throws the same error: # curl -k --cacert ./conf/nifi-cert.pem --cert ./conf/gs1_cert.pem --key ./conf/gs1_enckey.pem --pass ./conf/password https://nifi1.dev.cyhesion.com:9091/nifi curl: (58) unable to load client key: -8025 (SEC_ERROR_UNKNOWN_PKCS11_ERROR) I've tried single key/cert files and distinct ones. I've also converted the key into an RSA format one because there were some reports that keys that started "-----BEGIN ENCRYPTED PRIVATE KEY-----" sometimes case problems. It seems to be a difficulty with curl handling encrypted keys that have been created with OpenSSL. Curl doesn't return an error when I give it unencrypted certificates and keys. Minifi still doesn't work with unencrypted keys but I assume it didn't expect to be used in that way. I've played around with encryption algorithms as well, including ensuring the key is encrypted with triple-DES as suggested in stack-overflow but without success. As you pointed out curl on Centos 7 is bound to NSS rather than OpenSSL. Do you have any views on how this could be resolved? ... View more

Thanks for putting the effort in @mparisi. Yes, I used bootstrap and followed the prices from GitHub. But I left my build VM with a standard security profile rather than the heavier NIST 800-171 one. ... View more

A quick update @Timothy Spann and @mparisi. I've built from the current master on GitHub and am getting largely the same problems (just slightly different phrasing), which has led me to think again about what might be different about my environment. My standard centos VM is built with a NIST 800-171 security profile applied, which is where my thinking is at the moment. I'll have to look at that profile now to see what clamps may be causing the problem and gradually unwind them. Any ideas you may have would be great. It's going to be important to identify the blocker (if this is the cause) because many of my clients will have various security policies applied to the servers. ... View more

Thanks for your response and suggestions @Timothy Spann. I've tested the certs and they seem to be OK. I'm getting a self-signed response of 19 and when I have copied the CA cert into /etc/pki/ca-trust/source/anchors/ it responds 0(OK). Still not joy in getting things connected though. I'm about to start a fresh build from source in the hope that will change things. ... View more