- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Error in Base64EncodeContent processor after configuring repository encryption- protocol version 1
- Labels:
-
Apache NiFi
Created ‎08-11-2023 04:32 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have enabled the repository Encryption with protocol version 1, after that there is below error in Base64EncodeContent for some of the flowfiles and other flowfiles gets successfully processed.Can someone please help on this.
2023-08-08 19:26:07,368 ERROR [Timer-Driven Process Thread-5] o.a.n.p.standard.Base64EncodeContent Base64EncodeContent[id=7ba6e988-8bc9-392a-1ae8-f38733a69144] Processing halted: yielding [1 sec]
org.apache.nifi.processor.exception.MissingFlowFileException: Unable to find content for FlowFile
at org.apache.nifi.controller.repository.StandardProcessSession.handleContentNotFound(StandardProcessSession.java:3701)
at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3454)
at org.apache.nifi.processors.standard.Base64EncodeContent.onTrigger(Base64EncodeContent.java:135)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1360)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.nifi.controller.repository.ContentNotFoundException: Could not find content for StandardContentClaim [resourceClaim=StandardResourceClaim[id=1691502935403-13, container=default, section=13], offset=47426, length=13851]: Stream contained only 525 bytes but should have contained 904
at org.apache.nifi.controller.repository.io.FlowFileAccessInputStream.ensureAllContentRead(FlowFileAccessInputStream.java:49)
at org.apache.nifi.controller.repository.io.FlowFileAccessInputStream.read(FlowFileAccessInputStream.java:84)
at org.apache.nifi.controller.repository.io.TaskTerminationInputStream.read(TaskTerminationInputStream.java:68)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133)
at org.apache.nifi.processors.standard.util.ValidatingBase64InputStream.read(ValidatingBase64InputStream.java:37)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.nifi.processors.standard.util.ValidatingBase64InputStream.read(ValidatingBase64InputStream.java:53)
at org.apache.commons.codec.binary.BaseNCodecInputStream.read(BaseNCodecInputStream.java:173)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.nifi.processors.standard.Base64EncodeContent$2.process(Base64EncodeContent.java:141)
at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3436)
... 12 common frames omitted
Created on ‎08-14-2023 08:10 AM - edited ‎08-14-2023 08:18 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@tej_s @jakese
There are some recently discovered issues seen as more individuals start using the encrypted repository capability that have been addressed in Apache NiFi 1.23. I encourage you to see if you can reproduce your issue with release 1.23 or newer.
Some specific related fix that are part of 1.23:
https://issues.apache.org/jira/browse/NIFI-10235
https://issues.apache.org/jira/browse/NIFI-11670
11670 fixed some offset tracking issues and content handling issues that could very likely be causing your issue.
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt
Created ‎08-11-2023 08:27 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@tej_s
I recommend against switching to encrypted repositories while you still have content in your flowfile_repository, content_repository, and provenance_repository. What version of Apache NiFi are you using?
If you stop NiFi and delete the contents of these encrypted repositories and startup clean, do you encounter and issues like above with all new data being processed through your dataflows?
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt
Created ‎08-13-2023 11:24 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for your response!
The NiFi version that I am using is NiFi-1.17.0. However I tested the same with NiFi-1.15.0, NiFi-1.18.0 and NiFi-1.22.0. Still the same error occurs.
I configured the repository data encryption immediately after setting up the nifi when there was no data flowing through the system. Next time I tried with cleaning up the existing data in NiFi repositories and sending the new data still same issue persists.
Is there any other scenario that you think I should try and that might work. If yes please suggest that.
I am not sure, if this is any functional issue where Base64 decoding doesn't work with the AES GCM 256 encryption algorithm for repository encryption.
Created on ‎08-14-2023 08:10 AM - edited ‎08-14-2023 08:18 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@tej_s @jakese
There are some recently discovered issues seen as more individuals start using the encrypted repository capability that have been addressed in Apache NiFi 1.23. I encourage you to see if you can reproduce your issue with release 1.23 or newer.
Some specific related fix that are part of 1.23:
https://issues.apache.org/jira/browse/NIFI-10235
https://issues.apache.org/jira/browse/NIFI-11670
11670 fixed some offset tracking issues and content handling issues that could very likely be causing your issue.
If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.
Thank you,
Matt
Created ‎11-22-2023 03:25 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Upgrading NiFi to latest version NiFi-1.23.2 helped me on this issue and there are no issues with Base64EncodeContent processor after configuring repository encryption
@MattWho Thanks you!
Created ‎08-12-2023 08:21 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
this also happen to me
