Skip to main content
Dave0x1
Dave0x1
Gold ⭐️⭐️
January 19, 2024
Solved

javax.crypto.AEADBadTagException: Tag mismatch!

Hi there, I need some help regarding using a flow.xml.gz from 1.13 to 1.24. First our nifi setup: 

2 node cluster running on ec2 instances in aws where we run nifi in docker containers. We have succesfully upgraded from 1.13 to 1.24 after making a backup of the flow.xml.gz file.

We then stop the nifi containers and put back the flow.xml.gz from version 1.13. Then startup nifi. Unfortunately we get the following error:

Jan 19 07:20:58 nifi-node-1 docker[27934]: 2024-01-19 07:20:58,216 INFO [main] org.eclipse.jetty.server.Server Started @78240ms
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:58,304 INFO [main] org.apache.nifi.web.server.JettyServer Loading Flow...
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:58,326 INFO [main] o.apache.nifi.io.socket.SocketListener Now listening for connections from nodes on port 8888
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,036 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 10, col 20: cvc-complex-type.2.4.b: The content of element 'flowRegistry' is not complete. One of '{class}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,037 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 16, col 20: cvc-complex-type.2.4.b: The content of element 'flowRegistry' is not complete. One of '{class}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,037 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 27, col 16: cvc-complex-type.2.4.a: Invalid content was found starting with element 'value'. One of '{provided}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,037 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 33, col 16: cvc-complex-type.2.4.a: Invalid content was found starting with element 'value'. One of '{provided}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,045 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 39, col 16: cvc-complex-type.2.4.a: Invalid content was found starting with element 'value'. One of '{provided}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,045 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 45, col 19: cvc-complex-type.2.4.b: The content of element 'parameter' is not complete. One of '{provided}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,046 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 50, col 16: cvc-complex-type.2.4.a: Invalid content was found starting with element 'value'. One of '{provided}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,046 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 56, col 19: cvc-complex-type.2.4.b: The content of element 'parameter' is not complete. One of '{provided}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,046 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 61, col 16: cvc-complex-type.2.4.a: Invalid content was found starting with element 'value'. One of '{provided}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,047 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 67, col 19: cvc-complex-type.2.4.b: The content of element 'parameter' is not complete. One of '{provided}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,047 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 72, col 16: cvc-complex-type.2.4.a: Invalid content was found starting with element 'value'. One of '{provided}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,047 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 78, col 16: cvc-complex-type.2.4.a: Invalid content was found starting with element 'value'. One of '{provided}' is expected.
Jan 19 07:20:59 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,048 WARN [main] o.a.nifi.fingerprint.FingerprintFactory Schema validation error parsing Flow Configuration at line 84, col 16: cvc-complex-type.2.4.a: Invalid content was found starting with element 'value'. One of '{provided}' is expected.
Jan 19 07:21:00 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,703 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.
Jan 19 07:21:00 nifi-node-1 docker[27934]: org.apache.nifi.encrypt.EncryptionException: Decryption Failed with Algorithm [AES/GCM/NoPadding]
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:78)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.fingerprint.FingerprintFactory.decrypt(FingerprintFactory.java:996)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.fingerprint.FingerprintFactory.getLoggableRepresentationOfSensitiveValue(FingerprintFactory.java:605)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.fingerprint.FingerprintFactory.addParameter(FingerprintFactory.java:360)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.fingerprint.FingerprintFactory.addParameterContext(FingerprintFactory.java:326)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.fingerprint.FingerprintFactory.addFlowControllerFingerprint(FingerprintFactory.java:219)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.fingerprint.FingerprintFactory.createFingerprint(FingerprintFactory.java:155)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.fingerprint.FingerprintFactory.createFingerprint(FingerprintFactory.java:129)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.controller.inheritance.FlowFingerprintCheck.checkInheritability(FlowFingerprintCheck.java:45)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.controller.XmlFlowSynchronizer.sync(XmlFlowSynchronizer.java:205)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:42)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1530)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:817)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:457)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:896)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.NiFi.<init>(NiFi.java:172)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.NiFi.<init>(NiFi.java:83)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.NiFi.main(NiFi.java:332)
Jan 19 07:21:00 nifi-node-1 docker[27934]: Caused by: javax.crypto.AEADBadTagException: Tag mismatch!
Jan 19 07:21:00 nifi-node-1 docker[27934]: at java.base/com.sun.crypto.provider.GaloisCounterMode.decryptFinal(Unknown Source)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at java.base/com.sun.crypto.provider.CipherCore.finalNoPadding(Unknown Source)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at java.base/com.sun.crypto.provider.CipherCore.fillOutputBuffer(Unknown Source)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at java.base/com.sun.crypto.provider.CipherCore.doFinal(Unknown Source)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at java.base/com.sun.crypto.provider.AESCipher.engineDoFinal(Unknown Source)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at java.base/javax.crypto.Cipher.doFinal(Unknown Source)
Jan 19 07:21:00 nifi-node-1 docker[27934]: at org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:74)
Jan 19 07:21:00 nifi-node-1 docker[27934]: ... 18 common frames omitted
Jan 19 07:21:00 nifi-node-1 docker[27934]: 2024-01-19 07:20:59,733 INFO [Thread-0] org.apache.nifi.NiFi Application Server shutdown started
 
Any help would be much appreciated. Thank you.
Greetz
Dave
VidyaSargur
VidyaSargurAnswer
Community Manager
January 23, 2024

@Dave0x1, I'm happy to see that you resolved your issue. Can you kindly mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future?

Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
View in thread

6 replies

Dave0x1
Dave0x1Author
Gold ⭐️⭐️
January 19, 2024

After some trial and errors found where the problem lies, our nifi 1.13 uses the old alogrithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL

I tried to change this running the following command to change the flow.xml.gz and flow.json.gz to the correct algo.

# ./nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256

Java home: /opt/java/openjdk
NiFi home: /opt/nifi/nifi-current

Bootstrap Config File: /opt/nifi/nifi-current/conf/bootstrap.conf

Failed to process Flow Configuration [/nifi-data/conf/flow.xml.gz]
org.apache.nifi.encrypt.EncryptionException: Decryption Failed with Algorithm [AES/GCM/NoPadding]
at org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:78)
at org.apache.nifi.flow.encryptor.AbstractFlowEncryptor.getOutputEncrypted(AbstractFlowEncryptor.java:31)
at org.apache.nifi.flow.encryptor.XmlFlowEncryptor.processFlow(XmlFlowEncryptor.java:57)
at org.apache.nifi.flow.encryptor.StandardFlowEncryptor.processFlow(StandardFlowEncryptor.java:50)
at org.apache.nifi.flow.encryptor.command.FlowEncryptorCommand.processFlowConfiguration(FlowEncryptorCommand.java:135)
at org.apache.nifi.flow.encryptor.command.FlowEncryptorCommand.processFlowConfigurationFiles(FlowEncryptorCommand.java:119)
at org.apache.nifi.flow.encryptor.command.FlowEncryptorCommand.run(FlowEncryptorCommand.java:96)
at org.apache.nifi.flow.encryptor.command.SetSensitivePropertiesAlgorithm.main(SetSensitivePropertiesAlgorithm.java:29)
Caused by: javax.crypto.AEADBadTagException: Tag mismatch!
at java.base/com.sun.crypto.provider.GaloisCounterMode.decryptFinal(Unknown Source)
at java.base/com.sun.crypto.provider.CipherCore.finalNoPadding(Unknown Source)
at java.base/com.sun.crypto.provider.CipherCore.fillOutputBuffer(Unknown Source)
at java.base/com.sun.crypto.provider.CipherCore.doFinal(Unknown Source)
at java.base/com.sun.crypto.provider.AESCipher.engineDoFinal(Unknown Source)
at java.base/javax.crypto.Cipher.doFinal(Unknown Source)
at org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:74)
... 7 more
NiFi Properties Processed [/opt/nifi/nifi-current/conf/nifi.properties]

So i know here it's failing, but no solution yet.

Translate
    MattWho
    Titanium ⭐️⭐️⭐️
    MattWho
    Titanium ⭐️⭐️⭐️
    January 19, 2024

    @Dave0x1 

    That is a big jump in versions from 1.13 directly to 1.24.

    Use NiFi toolkit instead to change the algorithm.
    https://nifi.apache.org/download/
    NiFi Toolkit 1.24.0

     

    ./encrypt-config.sh -n <nifi.properties from original 1.13 NiFi> -f <flow.xml.gz from original 1.13 NiFi> -x -s <sensitive props key from NiFi> -b <bootstrap.conf from original 1.13 NiFi> -A NIFI_PBKDF2_AES_GCM_256 -g <new 1.24 flow.xml.gz filename>

     

    Then in your NiFi 1.24 remove or rename the current flow.xml.gz and flow.json.gz files.
    Place the flow.xml.gz output from above toolkit command into same location and make sure permissions and ownership are correct.

    Start your NiFi 1.24.  Since the flow.json.gz does not exist, NiFi will load the flow.xml.gz and upon successful startup generate the new flow.json.gz file it will load from that point forward each time NiFi is restarted.

    Hope this works for you.

    If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

    Thank you,
    Mat

    Translate
      Dave0x1
      Dave0x1Author
      Gold ⭐️⭐️
      January 22, 2024

      Thank you very much Mat for your advice. I'll try it out today and get back to you with the results. Sounds promising 🤓

      Translate
        Dave0x1
        Dave0x1Author
        Gold ⭐️⭐️
        January 22, 2024

        Hi Mat, it works! Awesome, nifi 1.24 starts up with the flow we had from 1.13 👏 Thank you very much for the advice. We are again one step further towards finalizing our migration plan.

        Only thing left is is now we get the following error from the flow with regards to the nifi-registry.

         

        Jan 22 10:57:46 awsinstancename docker[19956]: 2024-01-22 10:57:45,991 ERROR [Timer-Driven Process Thread-4] o.a.nifi.groups.StandardProcessGroup Failed to synchronize StandardProcessGroup[identifier=9311b848-f003-3b2a-ab11-af9e8884bc89,name=CheckScratchPrijs] with Flow Registry because could not retrieve version 7 of flow with identifier ab161029-c69f-4cf2-b6b3-12f444896e3e in bucket 3be9f87e-3a97-400c-a174-f335de21f23a Jan 22 10:57:46 awsinstancename docker[19956]: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.Alert.createSSLException(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.SSLHandshake.consume(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.TransportContext.dispatch(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source) Jan 22 10:57:46 awsinstancename docker[19956]: at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

        I'll open een separate thread for this

         

        Translate
        Dave0x1
        Dave0x1Author
        Gold ⭐️⭐️
        January 22, 2024

        Above registry issue also solve, with the help of this thread 🤓

        https://community.cloudera.com/t5/Support-Questions/NIfi-and-Nifi-Registry-Integration/m-p/286469

        Translate
          VidyaSargur
          Community Manager
          VidyaSargurAnswer
          Community Manager
          January 23, 2024

          @Dave0x1, I'm happy to see that you resolved your issue. Can you kindly mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future?

          Translate
          Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
          Terms & ConditionsAccessibility statement