Member since
07-30-2019
155
Posts
107
Kudos Received
33
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
7357 | 04-18-2019 08:27 PM | |
2454 | 12-31-2018 07:36 PM | |
4287 | 12-03-2018 06:47 PM | |
1381 | 06-02-2018 02:35 AM | |
3547 | 06-02-2018 01:30 AM |
08-15-2017
07:31 PM
Pierre Villard has written multiple tutorials about securing your NiFi instances with various authorizers. I would recommend following this Guide to Integrating NiFi and LDAP and refer to Secure Cluster Setup if necessary. The guide you were following originally was not designed to handle the LDAP certificates in your NiFi truststore, which is why you cannot make a secure connection to the LDAP server.
... View more
08-15-2017
06:37 PM
The error you pasted indicates that the truststore you configured for NiFi does not know any valid signers of the LDAP server's certificate. You can show this certificate by connecting to the LDAP server using OpenSSL s_client with the "-debug -state -showcerts" flags. Once you have that public certificate, you can examine the certificate chain and find an intermediate or root CA which signed it. You need to import that CA public certificate into the truststore you configure NiFi to use. "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target" is the error which indicates this.
... View more
08-14-2017
06:26 PM
2 Kudos
Alvin, The encrypted content repository feature is actively being worked on. As a rule, we cannot make claims about the delivery dates or versions of active development features. Hope this helps. If you have compliance requirements that sensitive data (such as PII, PCI/payment details, EPHI, etc.) is never stored on disk in plaintext, you can explore using the volatile content repository, but be aware there is the risk of data loss in the event of power failure, and this applies to all content objects, not just the sensitive records. With Apache NiFi 1.3.0, you can also use the RecordReader and RecordSetWriter approach -- while the EncryptedRecordReader and EncryptedRecordSetWriter controller services are not yet available, you could use a custom ScriptedRecordReader and ScriptedRecordSetWriter to decrypt and re-encrypt on the fly. The intermediate "record" object is never persisted to disk, so at no point would the plaintext data be written outside of volatile memory regardless of the content repository implementation.
... View more
08-11-2017
06:54 PM
Hi DH. As of Apache NiFi 1.2.0, only TLSv1.2 is supported for incoming connections [Release Notes - Security]. This is independent of the version of OpenSSL installed, and unrelated to the selection of TLS/SSL protocol in StandardSSLContextService (see discussion on PR 1986 about removing invalid protocol version options from the dropdown because they are not supported). However, because your client should be the same instance as your server (the SiteToSite reporting task is pointing back to itself, correct?) the same protocol versions and cipher suites should be available to both roles. Can you do a remote debugging session and trace the outgoing connection to determine what protocol versions and cipher suites are available to the SSLContext object at that time?
... View more
08-08-2017
12:19 AM
2 Kudos
What issue do you get when using
openssl s_client ? You should definitely be able to connect using s_client, although you may need to provide -CAfile option to trust the issued server certificate and -cert and -key options to provide a client certificate as below:
openssl s_client -connect nifi.nifi.apache.org:9443 -debug -state -CAfile conf/nifi-cert.pem -cert conf/client.pem -key conf/client.key
As for the handshake_failure issue, this may be because:
The certificate has expired or is invalid
The provided keystore or key password is incorrect and the controller service cannot access the keystore
The server and client cannot agree on a suitable cipher suite during negotiation
Diagnosing with s_client is definitely the correct tool. You can also enable TLS debugging via the bootstrap.conf file with the line:
java.arg.15=-Djavax.net.debug=ssl,handshake
This output will be in logs/nifi-bootstrap.log .
... View more
04-19-2017
05:38 PM
As noted in the REST API, clusterNodeId is a required field in the GET /provenance-events/{id} request when running in clustered mode. Use the Developer Tools of your browser to watch the requests that the UI fires when you get the details of a provenance event and you should see where to include the clusterNodeId in the request (my instance is not running in a cluster so it is not present in my request).
... View more
04-03-2017
06:27 PM
As @Hellmar Becker noted, SplitContent allows you to split on arbitrary byte sequences, but if you are looking for a specific word, SplitText will also achieve what you want. You may also want to look at RouteText , which allows you to apply a literal or regular expression to every line in the flowfile content and route each individually based on their matching results.
Finally, if you only care about the occurrence count of a specific word or sequence in the flowfile, you could use a small script in ExecuteScript or even ExecuteStreamCommand and use a terminal command like $ tr ' ' '\n' < FILE | grep WORD | wc -l (from here).
... View more
04-03-2017
05:00 PM
1 Kudo
The error is saying that one node in the cluster is attempting to make a heartbeat connection to the other node, but it is not providing a valid client certificate in order to authenticate itself during the TLS handshake negotiation. There are a few possible reasons for this error:
The node is not sending the client certificate. Ensure that nifi.security.needClientAuth=true and nifi.cluster.protocol.is.secure=true are present in your nifi.properties file.
The truststore on the receiving node does not contain the public key certificate of the connecting node. When you followed the instructions from that link, how did you generate the respective certificates? Using the Apache NiFi TLS Toolkit as described by Pierre should ensure that all node certificates are signed by the same CA and that the CA is imported into the common truststore. If you manually generated your certificates, ensure that they are trusted on each node (you can do this with OpenSSL's s_client tool).
... View more
04-03-2017
04:50 PM
You need to tell s_client that a self-signed certificate is ok by providing the -CAfile flag and the path to the CA's certificate, exported as PEM.
$ openssl s_client -connect HKLPATHAS03.example.com:9090 -CAfile /opt/certs/ca.pem
... View more
03-23-2017
06:51 PM
2 Kudos
Currently, NiFi supports encrypting/decrypting data through the EncryptContent processor, but the pre/post state of the data would still be stored in plaintext in the content repository. In general, transparent disk encryption/OS-level data encryption is recommended in conjunction with strict OS-level/POSIX access controls. There is a current effort to provide encrypted implementations of the flowfile (attribute), content, and provenance repositories. As Dan mentioned, a combination of encrypted payload and plaintext metadata for routing can work very well if the payload does not need to be processed/transformed inside NiFi.
... View more