Created 10-23-2023 09:52 AM
In Cloudera Data Flow, I'm using the `QueryDatabaseTable` processor with a maximum-value column to keep track of which records have already been processed. When I'm in Flow Designer testing the flow, I can see that it's maintaining state. Does this state persist if I deploy it, or does the deploy clear the state? I expect it would be the latter, but want to check.
Follow up question: Say I make changes to the flow and redeploy. Is the state cleared this time, or does it use the last deployments flow? I'd like to be confident that making changes to the flow doesn't cause it to reprocess all the data again.
Created 10-30-2023 06:50 AM
Hello @ipson first sorry my previous comment was not posted for some reason.
It is great to see you are exploring DataFlow Data Service on Cloudera Public Cloud.
To answer your question:
Does it persist state?
If you have to deploy a new version of the flow it means you are standing up a new namespace and in turn it will have a new zookeeper / NiFi / disk among other things contained within the kubernetes namespace.
Created 10-30-2023 06:50 AM
Hello @ipson first sorry my previous comment was not posted for some reason.
It is great to see you are exploring DataFlow Data Service on Cloudera Public Cloud.
To answer your question:
Does it persist state?
If you have to deploy a new version of the flow it means you are standing up a new namespace and in turn it will have a new zookeeper / NiFi / disk among other things contained within the kubernetes namespace.
Created 11-02-2023 01:35 PM
@ipson Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.
Regards,
Diana Torres,