Stateful processors in test flows vs. deployment

[#1]

In Cloudera Data Flow, I'm using the `QueryDatabaseTable` processor with a maximum-value column to keep track of which records have already been processed. When I'm in Flow Designer testing the flow, I can see that it's maintaining state. Does this state persist if I deploy it, or does the deploy clear the state? I expect it would be the latter, but want to check.

Follow up question: Say I make changes to the flow and redeploy. Is the state cleared this time, or does it use the last deployments flow? I'd like to be confident that making changes to the flow doesn't cause it to reprocess all the data again.

[#2 · Accepted solution]

Hello @ipson first sorry my previous comment was not posted for some reason.

It is great to see you are exploring DataFlow Data Service on Cloudera Public Cloud.
To answer your question:

Does it persist state?

Unfortunately there is currently no ability to change the flow on a running deployment, so you'd have to

• publish v2 to the catalog from flow designer
• terminate original deployment
• perform new deployment with v2

If you have to deploy a new version of the flow it means you are standing up a new namespace and in turn  it will have a new zookeeper / NiFi / disk among other things contained within the kubernetes namespace.

[#3]

@ipson Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.