Hi Andy, Your question is ambiguous -- are you looking to use NiFi to make an external HTTP REST request to some other service, and that service requires a username and password, or are you trying to perform behavior in NiFi (adding templates, process groups, etc.) and want to authenticate to NiFi in order to do this? For the former, you'll need to provide the credentials as an Authorization header via an attribute in the InvokeHTTP (or similar) processor. For the latter, you configure NiFi to use one or more authentication mechanisms -- client certificates, LDAP, Kerberos, or OpenID Connect. All but client certificates use a username/password combination, which you can then exchange for an access token as described in the first link you included. Once you have that auth token, you pass it on each request and NiFi verifies it to prove your identity. ... View more

You need the private key and public key to be stored in a Java Keystore (*.jks) file. You can import the PEM-encoded certificate and key into this form by using the following commands: openssl pkcs12 -export -in server.pem -inkey server.key -out server.p12 -name [some-alias] -chain keytool -importkeystore -deststorepass [yourpassword] -destkeypass [yourpassword] -destkeystore server.jks -srckeystore server.p12 -srcstoretype PKCS12 -srcstorepass [passwordfromabove] -alias [some-alias] When creating the temporary PKCS12 keystore, make sure to provide a password at the prompt, or the Java keytool utility will not accept it. Once you have the server.jks file, populate the properties as follows: Keystore file: path/to/server.jks Keystore password: [yourpassword] Key password: [yourpassword] Keystore type: JKS This will allow your NiFi instance/component to present a server certificate identifying itself and encrypt the channel. However, to connect to external HTTPS services, you will also need to provide a truststore. A truststore is a keystore file that contains only public certificates of other services to allow your system (in this case, NiFi) to trust them. If you have custom organizational certificates, you'll need to build your own truststore here. If you are just connecting to generic internet services, the JRE default should be fine: Truststore file: /Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre/lib/security/cacerts (your JRE path may be different) Truststore password: changeit Truststore type: JKS ... View more

The ExecuteScript processor does not support PHP. However, you can use ExecuteProcess or ExecuteStreamCommand to run the php command line application to execute a script. ... View more

Hi Laurent, Apache NiFi 1.4.0 uses Jetty 9.4.2 to provide the underlying web server, and Jetty after versions 9.4.0 only supports TLS v1.2 for incoming connections. I would recommend using a proxy with TLS termination which accepts incoming TLS v1.1 connections and re-establishing a connection to your NiFi service which uses TLS v1.2. * Ticket - NIFI-3361 Upgrade Jetty https://issues.apache.org/jira/browse/NIFI-3361 * Ticket - NIFI-3720 Update documentation for TLS protocol version changes https://issues.apache.org/jira/browse/NIFI-3720 * Jetty Documentation TLS and SSL Versions https://www.eclipse.org/jetty/documentation/current/configuring-ssl.html#tls-and-ssl-versions * Apache NiFi Release Notes for 1.2.0 noting TLS protocol version changes https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.2.0 * Apache NiFi Migration Guidance for 1.1.0 -> 1.2.0 noting the change https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance The actual announcement that Jetty changed the protocol versions supported is buried in their release notes somewhere. ... View more

Hi Prakash, I think there are multiple questions here so I've tried to split out my answers by my understanding. Please correct any erroneous assumptions I have made. Currently you can prepend the IV to the cipher text as the flowfile content to provide unique IV decryption via EncryptContent (see KeyedEncryptor L131). You are correct that ExecuteScript could also provide this behavior. I have included an example unit test which does this and the resulting log output to allow you to verify that. EncryptContent in encrypt mode automatically prepends the IV to the cipher text to allow the same processor in decrypt mode to handle this seamlessly. This is followed by a static 6 byte delimiter "NiFiIV" ( 4E 69 46 69 49 56 in hex) which is used by the decryptor to recognize the IV (see image). At this time, you cannot provide a static IV as a property descriptor in the EncryptContent processor. If this is desired functionality for you, you can open a Jira requesting a dynamic property to provide a static IV, but I do not believe it would be prioritized, as a static IV is not recommended best practice. I am not sure what this section means: The way, we are doing it in other applications is, we use the AES-CBC-PKC5Padding Keyed encryption along with a Initialization Vector and the Secret Key used, is encrypted using Standard PBE. I would like to know how to replicate this in NiFi. If you are using the combination of IV and raw secret key, there is no PBE (Password-Based Encryption) happening. If you are using a password, then some KDF (Key Derivation Function) is being employed. If you provide both the raw key value as a property in the EncryptContent processor and select " AES-CBC " as the encryption algorithm, the KDF property must be " None ". If you are able to use EncryptContent with the raw key hex value and it is succeeding, then you are already providing the IV in the correct format and location. Unit test for arbitrary IV with keyed encryption: @Test void testShouldAcceptArbitraryIVInContent() throws IOException { // Arrange final TestRunner testRunner = TestRunners.newTestRunner(new EncryptContent()) final String RAW_KEY_HEX = "ab" * 16 testRunner.setProperty(EncryptContent.RAW_KEY_HEX, RAW_KEY_HEX) testRunner.setProperty(EncryptContent.KEY_DERIVATION_FUNCTION, KeyDerivationFunction.NONE.name()) def keyedCipherEMs = EncryptionMethod.values().findAll { it.isKeyedCipher() } // Act keyedCipherEMs.each { EncryptionMethod encryptionMethod -> logger.info("Attempting {}", encryptionMethod.name()) testRunner.setProperty(EncryptContent.ENCRYPTION_ALGORITHM, encryptionMethod.name()) testRunner.setProperty(EncryptContent.MODE, EncryptContent.ENCRYPT_MODE) testRunner.enqueue(Paths.get("src/test/resources/hello.txt")) testRunner.clearTransferState() testRunner.run() testRunner.assertAllFlowFilesTransferred(EncryptContent.REL_SUCCESS, 1) MockFlowFile flowFile = testRunner.getFlowFilesForRelationship(EncryptContent.REL_SUCCESS).get(0) logger.info("Encrypted content of flowfile: ${Hex.encodeHexString(flowFile.data)}") logger.info("Unique IV: ${Hex.encodeHexString(flowFile.data)[0..31]}") testRunner.assertQueueEmpty() testRunner.setProperty(EncryptContent.MODE, EncryptContent.DECRYPT_MODE) testRunner.enqueue(flowFile) testRunner.clearTransferState() testRunner.run() // Assert testRunner.assertAllFlowFilesTransferred(EncryptContent.REL_SUCCESS, 1) logger.info("Successfully decrypted {}", encryptionMethod.name()) flowFile = testRunner.getFlowFilesForRelationship(EncryptContent.REL_SUCCESS).get(0) flowFile.assertContentEquals(new File("src/test/resources/hello.txt")) } } Log output from test: 1562 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Attempting AES_CBC 2675 [pool-2-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - An IV was provided of length 0 bytes for encryption but should be 16 bytes 2676 [pool-2-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - Generating new IV. The value can be obtained in the calling code by invoking 'cipher.getIV()'; 2680 [pool-2-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully encrypted FlowFile[0,hello.txt,38B] 2719 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Encrypted content of flowfile: af745bdb7fde42cf60dffd770eaa4f414e69466949565620ac7676b4ffee33668d0933523cf0 2724 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Unique IV: af745bdb7fde42cf60dffd770eaa4f41 2736 [pool-3-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully decrypted FlowFile[0,hello.txt,13B] 2737 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Successfully decrypted AES_CBC 2744 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Attempting AES_CTR 2750 [pool-4-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - An IV was provided of length 0 bytes for encryption but should be 16 bytes 2750 [pool-4-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - Generating new IV. The value can be obtained in the calling code by invoking 'cipher.getIV()'; 2750 [pool-4-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully encrypted FlowFile[1,hello.txt,35B] 2754 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Encrypted content of flowfile: 073764cb39ab8d2879c24fa516dc5ee04e6946694956c4d4c1ce83966bd0cb2554fc0e 2754 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Unique IV: 073764cb39ab8d2879c24fa516dc5ee0 2756 [pool-5-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully decrypted FlowFile[1,hello.txt,13B] 2757 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Successfully decrypted AES_CTR 2757 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Attempting AES_GCM 2763 [pool-6-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - An IV was provided of length 0 bytes for encryption but should be 16 bytes 2763 [pool-6-thread-1] WARN org.apache.nifi.security.util.crypto.AESKeyedCipherProvider - Generating new IV. The value can be obtained in the calling code by invoking 'cipher.getIV()'; 2765 [pool-6-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully encrypted FlowFile[2,hello.txt,51B] 2765 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Encrypted content of flowfile: 8c526b99b7d7534329c4a08cc345f9ba4e69466949568dab2fa15b5e940a2221d16659ca071b007fbff514009d0354515c604b 2765 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Unique IV: 8c526b99b7d7534329c4a08cc345f9ba 2767 [pool-7-thread-1] INFO org.apache.nifi.processors.standard.EncryptContent - EncryptContent[id=ba25adf1-2e42-41f8-a14f-c9b247360829] successfully decrypted FlowFile[2,hello.txt,13B] 2768 [main] INFO org.apache.nifi.processors.standard.TestEncryptContentGroovy - Successfully decrypted AES_GCM Process finished with exit code 0 ... View more

The flowfile content should be returned as the response body. Have you provided content on the incoming flowfile? ... View more

Using your browser's Developer Tools window, use the UI to clear a queue while monitoring the network tab. Everything the Apache NiFi UI does is performed via the REST API. You will be able to see exactly what requests are sent to the server to clear the connection queue and can recreate that programmatically. The specific API endpoint you want in this case is POST /flowfile-queues/{id}/drop-requests where {id} is the connection ID. ... View more

From my answer to this question on Stack Overflow: To extract the desired value, use the XPath expression //ErrorCode . This will return a String value -7. By selecting Destination flowfile-attribute, you can keep the flowfile content constant and put this new value in a flowfile attribute (i.e. named attribute ). You can chain the matched relationship to an UpdateAttribute processor which has the expression ${attribute:toNumber()} to convert it to a numerical representation, i.e. ${attribute:toNumber():plus(10)} would return 3. ... View more