Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Setting Initial Value of Stateful Variables in UpdateAttributes processor

avatar
Expert Contributor

This question has been asked before (here) but never answered so I'm hopeful that today is my lucky day and someone knows the answer.   Because unfortunately, the documentation does not cover the topic.

In the UpdateAttribute there is an option to Store State of an attribute by setting the value to "Store state locally".  But to do that I first need to set the "Stateful Variables Initial Value", but the documentation doesn't say how to do that, and everything I've tried fails.   The documentation provides an example for referencing an existing attribute variable, theCount, but there is no mention of how to initialize this attribute variable. 

I've attempted setting the initial value using:

  • theCount=1
  • {"theCount":1}
  • ${theCount=1}
  • 1
  • ${setStateValue('theCount')}     <- grasping at straws with this one

None of these work.  When I look at the value of the attribute theCount it always just shows:  "Empty string set".

I'm including a screen shot of my processor so you can see the configuration.  

ChuckE_0-1701401417557.png

Also, including a screenshot of the Attributes tab showing the theCount attribute.

ChuckE_1-1701401759347.png

 

 

 

 

2 ACCEPTED SOLUTIONS

avatar
Super Guru

@ChuckE,

This worked for me :

SAMSAL_0-1701437268401.png

UpdateAttribute Configuration:

SAMSAL_1-1701437310086.png

Notice the "Stateful Variables Initial Value" is set to 1

The UpdateAttribute success relationship has the result flowfile with the following attribute:

SAMSAL_2-1701437445671.png

If I run the flow again , the new flowfile will have the following attribute:

SAMSAL_3-1701437507671.png

Its strange that It did not work for you when set the initial value to 1 ? I dont see anything else wrong. If that still did not work for you, can you try to upgrade Nifi just to see if its not a bug with 1.19.1. I use 1.20 for testing but you can upgrade to the latest.

If you find this helpful please accept solution.

Thanks

 

 

 

View solution in original post

avatar
Super Guru

@ChuckE,

Regarding the state being persistent, actually you can clear the state by right click on the processor then select "View state" and then click on the "Clear state" link:

SAMSAL_0-1701460293820.png

This should reset the state to the initial value.

Regarding your second question of being able to initialize more than one variable, you can define as many stateful variables as you need in one processor, however they all can have one initial value. If you are looking to define different initial values for different stateful variables then you have to create a different UpdateAttribute processors for the variables that have common initial value. Another option - I never tried - if you want to utilize one processor then you can  use Advanced option to define rules that will set different initial value based on a common conidition, however you have to be careful how you set the first value on the first flowfile. For example, if you have two stateful variables Attr1 & Attr2 where the first flowfile  has Attr1 = 0 & Attr2 = 1  then increment both afterward, you can define the UpdateAttribute as follows :

SAMSAL_1-1701461079874.png

Notice how I set the Initial Value to Empty String since its required to set some value when using stateful.

Under Advanced I defined two rules: one to initialize Attr1 to 0  and another to initialize Attr2 to 1 when each is set to empty string :

Rule for Attr1:

SAMSAL_2-1701461446219.png

Rule for Attr2:

SAMSAL_3-1701461503236.png

Make sure to set the FlowFile Policy top right to "use original" , otherwise it will duplicate flowfiles for each matched rules if "clone" is used.

When setting the same attribute in Advanced  mode and in basic mode, the Advanced will take precedence if rules are met, so the first time the increment wont run.

The first flowfile will have the initial values since the rules are satisfied.

SAMSAL_4-1701461788304.png

The second flowfile the rules are not satisfied therefore the increment will happen:

SAMSAL_5-1701461859367.png

Depending on what you want to see on the first flowfile then you can adjust the initial values accordingly. Not sure if this will work for all scenarios but you can try, otherwise use different processor as I stated above. Also if anyone thinks that goes against best practices or might cause problem please advise.

If you find this helpful please accept solution.

Thanks

 

 

View solution in original post

7 REPLIES 7

avatar
Community Manager

Perhaps @MattWho or @stevenmatison will have some input here. 


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Super Guru

@ChuckE,

This worked for me :

SAMSAL_0-1701437268401.png

UpdateAttribute Configuration:

SAMSAL_1-1701437310086.png

Notice the "Stateful Variables Initial Value" is set to 1

The UpdateAttribute success relationship has the result flowfile with the following attribute:

SAMSAL_2-1701437445671.png

If I run the flow again , the new flowfile will have the following attribute:

SAMSAL_3-1701437507671.png

Its strange that It did not work for you when set the initial value to 1 ? I dont see anything else wrong. If that still did not work for you, can you try to upgrade Nifi just to see if its not a bug with 1.19.1. I use 1.20 for testing but you can upgrade to the latest.

If you find this helpful please accept solution.

Thanks

 

 

 

avatar
Expert Contributor

@SAMSAL 
Thank you for the response.  You are correct, this does work.  I created a new flow from scratch and it worked.  BUT, what I've noticed is that when you set the state on this processor it makes the state VERY STICKY.  You can't change anything in this processor once it has been initialized and then ran.  Even if you turn the processor off, it still remembers the state.  I wasn't aware that the state was that persistent and sticky.  So to make changes to the attribute variable you need to use and configure a new processor.

The next question then is how do you initialize more than one attribute variable?  

avatar
Master Mentor

@ChuckE 

You can right click on the UpdateAttribute processor and "view state" which will allow you to clear the stored state for that processor without needing to create a new processor.

Every dynamic property you add becomes a different local state variable that is added to the FlowFiles as a FlowFile Attribute.

Thanks,
Matt

avatar
Super Guru

@ChuckE,

Regarding the state being persistent, actually you can clear the state by right click on the processor then select "View state" and then click on the "Clear state" link:

SAMSAL_0-1701460293820.png

This should reset the state to the initial value.

Regarding your second question of being able to initialize more than one variable, you can define as many stateful variables as you need in one processor, however they all can have one initial value. If you are looking to define different initial values for different stateful variables then you have to create a different UpdateAttribute processors for the variables that have common initial value. Another option - I never tried - if you want to utilize one processor then you can  use Advanced option to define rules that will set different initial value based on a common conidition, however you have to be careful how you set the first value on the first flowfile. For example, if you have two stateful variables Attr1 & Attr2 where the first flowfile  has Attr1 = 0 & Attr2 = 1  then increment both afterward, you can define the UpdateAttribute as follows :

SAMSAL_1-1701461079874.png

Notice how I set the Initial Value to Empty String since its required to set some value when using stateful.

Under Advanced I defined two rules: one to initialize Attr1 to 0  and another to initialize Attr2 to 1 when each is set to empty string :

Rule for Attr1:

SAMSAL_2-1701461446219.png

Rule for Attr2:

SAMSAL_3-1701461503236.png

Make sure to set the FlowFile Policy top right to "use original" , otherwise it will duplicate flowfiles for each matched rules if "clone" is used.

When setting the same attribute in Advanced  mode and in basic mode, the Advanced will take precedence if rules are met, so the first time the increment wont run.

The first flowfile will have the initial values since the rules are satisfied.

SAMSAL_4-1701461788304.png

The second flowfile the rules are not satisfied therefore the increment will happen:

SAMSAL_5-1701461859367.png

Depending on what you want to see on the first flowfile then you can adjust the initial values accordingly. Not sure if this will work for all scenarios but you can try, otherwise use different processor as I stated above. Also if anyone thinks that goes against best practices or might cause problem please advise.

If you find this helpful please accept solution.

Thanks

 

 

avatar
Expert Contributor

You are awesome @SAMSAL Thanks so much for the great information.  This helps me a lot. 

Thank goodness for this community, because I think the documentation is nearly adequate.

avatar
Master Mentor

@ChuckE 
Plus 1 on @SAMSAL post.  I also had no problem with it working on my CFM 2.1.5.3 SP3 (Based off Apache NiFi 1.18 with many bug fixes from 1.19 - 1.20) 

MattWho_0-1701446677533.png

MattWho_1-1701446775604.png

 

 


Thanks,
Matt