Member since
07-30-2019
155
Posts
106
Kudos Received
33
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2960 | 04-18-2019 08:27 PM | |
952 | 12-31-2018 07:36 PM | |
1840 | 12-03-2018 06:47 PM | |
652 | 06-02-2018 02:35 AM | |
1697 | 06-02-2018 01:30 AM |
03-21-2018
06:28 PM
1 Kudo
Veerendra,
Please enable
DEBUG level logging by editing the $NIFI_HOME/conf/logback.xml file and adding these lines:
<logger name="org.apache.nifi.processors.standard.EncryptContent" level="DEBUG"/>
<logger name="org.apache.nifi.security.util.crypto" level="DEBUG"/>
Wait 30 seconds for the changes to be picked up by NiFi and then please try to trigger the same flow. You should get more verbose output in the log file.
The sequence "Failed to initialize decryption algorithm because" isn't present anywhere in NiFi code, so I imagine this is coming from an underlying library, probably BouncyCastle. Can you please share the configuration you're using for EncryptContent processor and some sample input data (preferably generated specifically for debugging so it does not contain any sensitive data).
... View more
03-02-2018
05:50 PM
The .p12 is the client certificate and does contain the private key to identify the user (it must contain the private key in order to actually identify the user and perform any encryption/signing during TLS mutual authentication). Don't import the .p12 into any server; it is just for the browser to authenticate the user. The public key of the CA is written out in nifi-cert.pem, and the various truststores from the host directories (i.e. <NiFi Instance 2 IP>/truststore.jks) contain that as a trustedCertEntry already. You can verify this by running keytool -list -verbose -keystore truststore.jks.
... View more
02-20-2018
08:02 PM
When you configure the RPG, use the URL for NiFi that contains the port the API/UI is running on (in your case, 8443, not 8899). NiFi will negotiate the RAW/HTTP(S) S2S port automatically over that communication. Do NOT import the private key of any server to any other server or truststore. The private key remains only on the instance you intend to secure with it. The public key can (and must) be imported to the other/remote truststores unless the key pairs were all signed by the same root, in which case the root public key alone in the truststore is sufficient.
... View more
02-15-2018
08:56 PM
For Site to Site, you configure the remote process group on the "originating"/"source" NiFi to point to the URL of the "remote"/"destination" NiFi and it will negotiate the rest on its own. So in your RPG, you would provide the URL "https://<instance 1 IP>:8443/nifi" assuming 8443 is the port that the NiFi UI/API is running on. You can also secure RAW transport connections as well -- it simply uses a secure socket connection. There is more information contained in the following resources: * Apache NiFi Admin Guide - Site to Site Properties * Apache NiFi User Guide - Site to Site * Apache NiFi Feature Description -- HTTP(S) for Site to Site (documents existing RAW capability as well)
... View more
02-15-2018
08:43 PM
The error you are getting indicates that the server certificate presented by the other NiFi instance has a Subject Alternative Name that is different from the expected hostname. You'll need to add the hostname into the list of SANs. This is a newer policy described by RFC 6125.
... View more
02-14-2018
06:45 PM
1 Kudo
Bryan Bende has written a good tutorial here. Do you have a specific error message or failure you can share to help diagnose your issue?
... View more
02-08-2018
12:15 AM
Try removing the users.xml and authorizations.xml files (back these up if you have made other changes prior to securing the application). These files are generated and populated if they are not present at application startup, which means they will be auto-generated the first time the application is run, but not re-generated automatically once they exist. Ensure your files look something like the following:
users.xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
<groups/>
<users>
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c" identity="CN=alopresto, OU=Apache NiFi"/>
</users>
</tenants>
authorizers.xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizers>
<authorizer>
<identifier>file-provider</identifier>
<class>org.apache.nifi.authorization.FileAuthorizer</class>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Users File">./conf/users.xml</property>
<property name="Initial Admin Identity">CN=alopresto, OU=Apache NiFi</property>
<property name="Legacy Authorized Users File"></property>
</authorizer>
</authorizers>
authorizations.xml <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizations>
<policies>
<policy identifier="0f4d2dd0-7c8b-39ce-8292-0e459a8dd69f" resource="/flow" action="R">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="9f665453-1e5f-36b4-8f4c-e125491c1fd2" resource="/data/process-groups/4aa66ece-015a-1000-4348-efd94f63c4df" action="R">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="38e2a10c-a75f-3b48-8954-6e5af69c837b" resource="/data/process-groups/4aa66ece-015a-1000-4348-efd94f63c4df" action="W">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="67684272-2833-32e6-954a-d33b7dfc80b2" resource="/process-groups/4aa66ece-015a-1000-4348-efd94f63c4df" action="R">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="ce02741b-adb1-37cf-a818-7d5476e49805" resource="/process-groups/4aa66ece-015a-1000-4348-efd94f63c4df" action="W">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="2148640b-305c-3c53-8a92-5111159729a1" resource="/restricted-components" action="W">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="cc00676b-e0bb-34be-8ae5-89a35dd3ebcb" resource="/tenants" action="R">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="2bf2a497-4045-37d6-8a7f-e8b2753da197" resource="/tenants" action="W">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="9abc4f3b-008c-3ad2-81d3-85410b435c2e" resource="/policies" action="R">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="f4d3b871-7144-3d22-b93d-67110a369647" resource="/policies" action="W">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="e6cdf807-d293-3c5c-8123-0b42c4370a7e" resource="/controller" action="R">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="03947efb-1e7c-3c42-90ac-ba0ebd1e64fd" resource="/controller" action="W">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
<policy identifier="015a1025-3be1-13fc-af08-0f6bf4e9285a" resource="/provenance" action="R">
<user identifier="69d36646-d7cd-33ef-83a7-54d95f67214c"/>
</policy>
</policies>
</authorizations>
... View more
02-07-2018
06:36 PM
It appears that you are specifying a different DN string in your command-line invocation of the TLS Toolkit and your initial admin identity: tls-toolkit.bat standalone -n localhost -C "CN=admin,OU=NIFI"
<property name="Initial Admin Identity">CN=admin, OU=NIFI</property> There is a space between "admin," and "OU=NIFI" in the IAI property definition, but not in the certificate generation command. These strings need to match for NiFi to authenticate the user via client certificate/LDAP/Kerberos.
... View more
01-19-2018
04:03 AM
Hi Alvin, As stated above, I cannot indicate prioritization or scheduling of feature delivery. I am eager to develop this feature, as I am sure many users would like it to be available as well. You can always monitor activity on the Apache NiFi Jira and the mailing lists.
... View more
11-30-2017
07:28 PM
It depends on a number of factors: Apache NiFi version -- < 1.2.0, NiFi will support all SSL and TLS protocol versions. After 1.2.0, only TLS v1.2 is supported for incoming connections (i.e. REST API, UI, ListenHTTP processor, etc.), while the other protocol versions are supported for outgoing connections (i.e. InvokeHTTP connection to a legacy server). Java Runtime Environment -- Java 8 Update 31 and Java 7 Update 75 both disable support for SSLv2 and SSLv3 protocols. If you are running on a JRE later than this, you need to explicitly enable SSL (against best practices) by modifying the java.security file or using static initialization. Again, this is a very bad idea. The best solution here is to upgrade the protocol version you are using to TLS v1.2 if possible. If not, introduce a proxy server to perform bidirectional handshakes. NiFi/HDF is not designed to allow legacy SSL protocols due to their demonstrated security vulnerabilities.
... View more
11-20-2017
06:44 PM
Hi Andy, Your question is ambiguous -- are you looking to use NiFi to make an external HTTP REST request to some other service, and that service requires a username and password, or are you trying to perform behavior in NiFi (adding templates, process groups, etc.) and want to authenticate to NiFi in order to do this? For the former, you'll need to provide the credentials as an Authorization header via an attribute in the InvokeHTTP (or similar) processor. For the latter, you configure NiFi to use one or more authentication mechanisms -- client certificates, LDAP, Kerberos, or OpenID Connect. All but client certificates use a username/password combination, which you can then exchange for an access token as described in the first link you included. Once you have that auth token, you pass it on each request and NiFi verifies it to prove your identity.
... View more
11-17-2017
11:00 PM
Please provide the version of NiFi/HDF you are using, the configuration of the StandardSSLContextService, and the full stacktrace from nifi-app.log. The information above is not enough for us to diagnose this issue.
... View more
11-08-2017
12:50 AM
1 Kudo
You need the private key and public key to be stored in a Java Keystore (*.jks) file. You can import the PEM-encoded certificate and key into this form by using the following commands: openssl pkcs12 -export -in server.pem -inkey server.key -out server.p12 -name [some-alias] -chain
keytool -importkeystore -deststorepass [yourpassword] -destkeypass [yourpassword] -destkeystore server.jks -srckeystore server.p12 -srcstoretype PKCS12 -srcstorepass [passwordfromabove] -alias [some-alias] When creating the temporary PKCS12 keystore, make sure to provide a password at the prompt, or the Java keytool utility will not accept it. Once you have the server.jks file, populate the properties as follows: Keystore file: path/to/server.jks Keystore password: [yourpassword] Key password: [yourpassword] Keystore type: JKS This will allow your NiFi instance/component to present a server certificate identifying itself and encrypt the channel. However, to connect to external HTTPS services, you will also need to provide a truststore. A truststore is a keystore file that contains only public certificates of other services to allow your system (in this case, NiFi) to trust them. If you have custom organizational certificates, you'll need to build your own truststore here. If you are just connecting to generic internet services, the JRE default should be fine: Truststore file: /Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre/lib/security/cacerts (your JRE path may be different) Truststore password: changeit Truststore type: JKS
... View more
11-04-2017
01:29 AM
Hans, This is the data as it arrives and you need to transform it into SQL INSERT statements? I spent the afternoon building a ScriptedRecordSetWriter in Groovy to do the translation (and it works), but Matt Burgess informed me that it was even easier than that. You can use PutDatabaseRecord with a JsonTreeReader to automatically translate the incoming JSON into a record and then create an INSERT statement for each. Try that and let me know if it doesn't solve your issue.
... View more
11-03-2017
06:43 PM
Hi Hans, The record process is definitely going to be more performant for you in this case. You should be able to use ConvertRecord with a JsonTreeReader and a ScriptedRecordSetWriter to convert each JSON record into a SQL statement directly. Can you provide your reader and writer definitions and some sample input and desired output so we can help craft the correct configurations?
... View more
11-03-2017
12:15 AM
Hi Sai, I believe the code that is failing is because the 1.2.0 version of Apache NiFi used POSIX file permissions to secure the sensitive.key file. However, Windows has trouble supporting those Java methods. Please file a Jira here and fully document the version of the Operating System, Java, and NiFi you are using.
... View more
10-26-2017
12:40 AM
Alex, your question regarding the difference between option 1 and option 5 is a good one. Option 1 discusses the readers and writers that support the provenance repository implementation. These classes serialize and deserialize provenance events from Java objects to byte streams which can be written to the repository files on disk. Option 5 references the record readers and writers which are used on the NiFi canvas to support the abstract "record" concept of a collection of individual units of data within a single flowfile. In this case, the reader and writer classes convert data between external formats (JSON, CSV, arbitrary via
ScriptedRecord* ) and the NiFi internal record format. I understand these concepts seem related, but these classes are completely separate and there is no overlap whatsoever. The as-yet-undeveloped EncryptedRecordReader and EncryptedRecordSetWriter classes you mention would allow you to operate on encrypted flowfile content, i.e. a flowfile contained 100 lines of customer data and some of the column values were PII/PCI/PHI and therefore encrypted. If you needed to update these records (let's say add a new property to each record which contained the last four digits of a credit card number, but the full number value was encrypted), you could use an UpdateRecord processor as follows:
EncryptedRecordReader to decrypt the records ephemerally
Add a property "lastFourDigits" which reads the /PAN field and slices the last four digits
EncryptedRecordSetWriter would re-encrypt the sensitive fields
All of these actions happen within the lifecycle of a single
@OnTrigger of the UpdateRecord processor (even though some logic is being performed by the controller services), so none of the plaintext "record" data is persisted anywhere on the system (it is only in RAM). To be clear, in this situation, the actual implementation of the record reader and writer would need to combine the crypto capabilities with the format, so it would actually be something like EncryptedJsonRecordReader and EncryptedJsonRecordSetWriter . I haven't done the full architecture work here yet, but obviously it's not ideal to have 2*n implementations just to provide the crypto capabilities. It is likely this would require architecture changes, either to allow multiple sequentially stacked readers and writers in the processor, or the Encrypted* implementations would accept a "type-specific" record reader/writer in their definitions and perform this task via composition. That way you would maintain the type-conversion flexibility that currently exists (i.e. EncryptedRecordReader has a JsonRecordReader and the EncryptedRecordSetWriter has a CsvRecordSetWriter , etc.).
As for when NiFi persists data to disk, this is usually done during/after the
@OnTrigger phase completes in the data lifecycle. You can see code like flowFile = processSession.putAttribute(flowFile, JMS_SOURCE_DESTINATION_NAME, destinationName); or
flowFile = processSession.write(flowFile, new OutputStreamCallback() { @Override public void process(final OutputStream out) throws IOException { out.write(response.getMessageBody()); } });
This is populating the flowfile attributes or content respectively, and then on
processSession.commit() that data is persisted to whichever implementation of the content/flowfile repository that is configured (i.e. could be written to disk, volatile memory, etc.) There is a good document which goes into depth on the write-ahead log implementation, and I wrote extensively about the provenance repository serialization here. I hope this clarifies the system. Please follow up if you have further questions.
... View more
10-25-2017
10:33 PM
Hi Suraj,
Can you please confirm that your nifi.properties is configured to use the LDAP Identity Provider for authentication? Client certificate authentication is always enabled, so converting a the NiFi JKS keystore to PKCS12 and loading it in your browser (while not recommended as a best practice) would allow you to authenticate as an identity trusted by the same certificate which signed the NiFi server certificate, which I believe is what you are describing.
In addition to configuring the login-identity-providers.xml file as you have above, you need to instruct NiFi to allow LDAP login by populating the following value in nifi.properties :
nifi.security.user.login.identity.provider=ldap-provider
If you have done this as well, please share the content of $NIFI_HOME/logs/nifi-user.log and $NIFI_HOME/logs/nifi-app.log to help us diagnose the issue. Thanks.
... View more
10-21-2017
07:05 PM
The ExecuteScript processor does not support PHP. However, you can use ExecuteProcess or ExecuteStreamCommand to run the php command line application to execute a script.
... View more
10-12-2017
07:09 PM
If you are using client certificates as an authentication mechanism, you will need to provide a valid client certificate in the Java invocation of the API. You can do this by loading a Java Keystore (*.jks) file containing the private key of the client certificate and presenting that. An example of the code to do so (using Apache httpclient and JRE variables) is here. You can also do this explicitly by writing code to parse an arbitrary keystore and load a key via SSLContext and SSLSockets (see here).
... View more
10-11-2017
09:54 PM
Laurent, feel free to contact me directly at alopresto@apache.org for further discussion.
... View more
10-11-2017
04:57 PM
1 Kudo
Hi Laurent, Apache NiFi 1.4.0 uses Jetty 9.4.2 to provide the underlying web server, and Jetty after versions 9.4.0 only supports TLS v1.2 for incoming connections. I would recommend using a proxy with TLS termination which accepts incoming TLS v1.1 connections and re-establishing a connection to your NiFi service which uses TLS v1.2. * Ticket - NIFI-3361 Upgrade Jetty https://issues.apache.org/jira/browse/NIFI-3361 * Ticket - NIFI-3720 Update documentation for TLS protocol version changes https://issues.apache.org/jira/browse/NIFI-3720 * Jetty Documentation TLS and SSL Versions https://www.eclipse.org/jetty/documentation/current/configuring-ssl.html#tls-and-ssl-versions * Apache NiFi Release Notes for 1.2.0 noting TLS protocol version changes https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.2.0 * Apache NiFi Migration Guidance for 1.1.0 -> 1.2.0 noting the change https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance The actual announcement that Jetty changed the protocol versions supported is buried in their release notes somewhere.
... View more
10-03-2017
06:50 PM
1 Kudo
Hi Prakash,
I think there are multiple questions here so I've tried to split out my answers by my understanding. Please correct any erroneous assumptions I have made.
Currently you can prepend the IV to the cipher text as the flowfile content to provide unique IV decryption via EncryptContent (see KeyedEncryptor L131). You are correct that ExecuteScript could also provide this behavior. I have included an example unit test which does this and the resulting log output to allow you to verify that. EncryptContent in encrypt mode automatically prepends the IV to the cipher text to allow the same processor in decrypt mode to handle this seamlessly. This is followed by a static 6 byte delimiter "NiFiIV" ( 4E 69 46 69 49 56 in hex) which is used by the decryptor to recognize the IV (see image).
At this time, you cannot provide a static IV as a property descriptor in the EncryptContent processor. If this is desired functionality for you, you can open a Jira requesting a dynamic property to provide a static IV, but I do not believe it would be prioritized, as a static IV is not recommended best practice.
I am not sure what this section means:
The way, we are doing it in other applications is, we use the AES-CBC-PKC5Padding Keyed encryption along with a Initialization Vector and the Secret Key used, is encrypted using Standard PBE. I would like to know how to replicate this in NiFi.
If you are using the combination of IV and raw secret key, there is no
PBE (Password-Based Encryption) happening. If you are using a password, then some KDF (Key Derivation Function) is being employed. If you provide both the raw key value as a property in the EncryptContent processor and select " AES-CBC " as the encryption algorithm, the KDF property must be " None ".
If you are able to use EncryptContent with the raw key hex value and it is succeeding, then you are already providing the IV in the correct format and location.
Unit test for arbitrary IV with keyed encryption:
@Test
void testShouldAcceptArbitraryIVInContent() throws IOException {
// Arrange
final TestRunner testRunner = TestRunners.newTestRunner(new EncryptContent())
final String RAW_KEY_HEX = "ab" * 16
testRunner.setProperty(EncryptContent.RAW_KEY_HEX, RAW_KEY_HEX)
testRunner.setProperty(EncryptContent.KEY_DERIVATION_FUNCTION, KeyDerivationFunction.NONE.name())
def keyedCipherEMs = EncryptionMethod.values().findAll { it.isKeyedCipher() }
// Act
keyedCipherEMs.each { EncryptionMethod encryptionMethod ->
logger.info("Attempting {}", encryptionMethod.name())
testRunner.setProperty(EncryptContent.ENCRYPTION_ALGORITHM, encryptionMethod.name())
testRunner.setProperty(EncryptContent.MODE, EncryptContent.ENCRYPT_MODE)
testRunner.enqueue(Paths.get("src/test/resources/hello.txt"))
testRunner.clearTransferState()
testRunner.run()
testRunner.assertAllFlowFilesTransferred(EncryptContent.REL_SUCCESS, 1)
MockFlowFile flowFile = testRunner.getFlowFilesForRelationship(EncryptContent.REL_SUCCESS).get(0)
logger.info("Encrypted content of flowfile: ${Hex.encodeHexString(flowFile.data)}")
logger.info("Unique IV: ${Hex.encodeHexString(flowFile.data)[0..31]}")
testRunner.assertQueueEmpty()
testRunner.setProperty(EncryptContent.MODE, EncryptContent.DECRYPT_MODE)
testRunner.enqueue(flowFile)
testRunner.clearTransferState()
testRunner.run()
// Assert
testRunner.assertAllFlowFilesTransferred(EncryptContent.REL_SUCCESS, 1)
logger.info("Successfully decrypted {}", encryptionMethod.name())
flowFile = testRunner.getFlowFilesForRelationship(EncryptContent.REL_SUCCESS).get(0)
flowFile.assertContentEquals(new File("src/test/resources/hello.txt"))
}
}
Log output from test:
1562 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Attempting AES_CBC
2675 [pool-2-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - An IV was provided of length 0 bytes for encryption but should be 16 bytes
2676 [pool-2-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - Generating new IV. The value can be obtained in the calling code by invoking 'cipher.getIV()';
2680 [pool-2-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully encrypted FlowFile[0,hello.txt,38B]
2719 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Encrypted content of flowfile: af745bdb7fde42cf60dffd770eaa4f414e69466949565620ac7676b4ffee33668d0933523cf0
2724 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Unique IV: af745bdb7fde42cf60dffd770eaa4f41
2736 [pool-3-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully decrypted FlowFile[0,hello.txt,13B]
2737 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Successfully decrypted AES_CBC
2744 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Attempting AES_CTR
2750 [pool-4-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - An IV was provided of length 0 bytes for encryption but should be 16 bytes
2750 [pool-4-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - Generating new IV. The value can be obtained in the calling code by invoking 'cipher.getIV()';
2750 [pool-4-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully encrypted FlowFile[1,hello.txt,35B]
2754 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Encrypted content of flowfile: 073764cb39ab8d2879c24fa516dc5ee04e6946694956c4d4c1ce83966bd0cb2554fc0e
2754 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Unique IV: 073764cb39ab8d2879c24fa516dc5ee0
2756 [pool-5-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully decrypted FlowFile[1,hello.txt,13B]
2757 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Successfully decrypted AES_CTR
2757 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Attempting AES_GCM
2763 [pool-6-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - An IV was provided of length 0 bytes for encryption but should be 16 bytes
2763 [pool-6-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - Generating new IV. The value can be obtained in the calling code by invoking 'cipher.getIV()';
2765 [pool-6-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully encrypted FlowFile[2,hello.txt,51B]
2765 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Encrypted content of flowfile: 8c526b99b7d7534329c4a08cc345f9ba4e69466949568dab2fa15b5e940a2221d16659ca071b007fbff514009d0354515c604b
2765 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Unique IV: 8c526b99b7d7534329c4a08cc345f9ba
2767 [pool-7-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully decrypted FlowFile[2,hello.txt,13B]
2768 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Successfully decrypted AES_GCM
Process finished with exit code 0
... View more
08-27-2017
07:56 PM
Use a ReplaceText processor to replace the content of the incoming flowfile with the static response body. Because you can provide multiple outgoing connections from the previous processor, you will not lose the flowfile content, but this is the way to control the response via HandleHTTPResponse .
... View more
08-25-2017
06:40 PM
1 Kudo
The flowfile content should be returned as the response body. Have you provided content on the incoming flowfile?
... View more
08-25-2017
06:38 PM
Using your browser's Developer Tools window, use the UI to clear a queue while monitoring the network tab. Everything the Apache NiFi UI does is performed via the REST API. You will be able to see exactly what requests are sent to the server to clear the connection queue and can recreate that programmatically. The specific API endpoint you want in this case is POST /flowfile-queues/{id}/drop-requests where {id} is the connection ID.
... View more
08-23-2017
07:51 PM
D H, You bring up a good point that the EKU restriction should probably be made more evident in the docs. It makes sense -- the certificates are used in both server and client identification roles, so the roles must both be present if either is populated, but many people are unfamiliar with the EKU behavior (or unaware that the certificates serve dual purposes). I hope you are able to resolve your issue quickly.
... View more
08-23-2017
07:45 PM
1 Kudo
From
my answer to this question on Stack Overflow:
To extract the desired value, use the XPath expression
//ErrorCode . This will return a String value -7. By selecting Destination flowfile-attribute, you can keep the flowfile content constant and put this new value in a flowfile attribute (i.e. named attribute ).
You can chain the matched relationship to an UpdateAttribute processor which has the expression ${attribute:toNumber()} to convert it to a numerical representation, i.e. ${attribute:toNumber():plus(10)} would return 3.
... View more
08-15-2017
07:31 PM
Pierre Villard has written multiple tutorials about securing your NiFi instances with various authorizers. I would recommend following this Guide to Integrating NiFi and LDAP and refer to Secure Cluster Setup if necessary. The guide you were following originally was not designed to handle the LDAP certificates in your NiFi truststore, which is why you cannot make a secure connection to the LDAP server.
... View more
08-15-2017
06:37 PM
The error you pasted indicates that the truststore you configured for NiFi does not know any valid signers of the LDAP server's certificate. You can show this certificate by connecting to the LDAP server using OpenSSL s_client with the "-debug -state -showcerts" flags. Once you have that public certificate, you can examine the certificate chain and find an intermediate or root CA which signed it. You need to import that CA public certificate into the truststore you configure NiFi to use. "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target" is the error which indicates this.
... View more