question Re: Reset the Filename sequence no./counter everyday in Support Questions

Reset the Filename sequence no./counter everyday

techNerd — Wed, 11 Oct 2023 13:40:38 GMT

Hi,

Every 5 min i have a file generated from the NiFi Processor GenerateFlowFile:

filename_0.xml

filename_1.xml

filename_2.xml

....

I would like to asked how do i reset the sequence no. everyday ?

Thanks. Would appreciate help with example and explanation.

The first method i found is using nextInt():

the second method is using seq:

Please advise me with steps and example. thanks

Re: Reset the Filename sequence no./counter everyday

drewski7 — Wed, 11 Oct 2023 17:53:04 GMT

Maybe you could reset the state via NiFi-REST-API at beginning of your flow or separately on a schedule cron every morning

Could be:
POST "https://[ip:port]/nifi-api/processors/${processor-id}/state/clear-requests"

This is the request which NiFi itself uses when you go to the processor in the UI and choose the menu-option "view state" -> "clear state".

Re: Reset the Filename sequence no./counter everyday

techNerd — Wed, 11 Oct 2023 23:42:52 GMT

Hi @drewski7,

Thanks for the replying.

From your reply, I have added in NiFi InvokeHTTP processor which will automatically help to reset my seq no. counter every morning. I am using the second method above which using seq no. which store state locally.

I have set the HTTP Method to POST and HTTP URL to https://[ip:port]/nifi-api/processors/${processor-id}/state/clear-requests

I have scheduled the NiFi InvokeHTTP Processor to run every morning 00:00 using CRON driven Scheduling Strategy

Below image are the NiFi UpdateAttribute Processor which show the processor ID that i used inside my HTTP URL

Just to check did i did it correctly as what you mention in your reply ?

I tried to change my computer time to 11:55 pm and let it run to 00:00 the next day, but it does not invokeHTTP POST request once it reach 00:00. I might have made some mistake somewhere.

appreciate if you could advise me with example and explanation. Thanks a lot.

Re: Reset the Filename sequence no./counter everyday

techNerd — Thu, 12 Oct 2023 15:55:06 GMT

Hi @drewski7 ,

I noticed that the invokeHTTP processor will only POST request to clear state when the updateAttribute processor which contain the state is STOP.

Error Message:

i got from invokeHTTP processor "UpdateAttribute[id=e6c537bd-c24b-1801-6fdd-b37c23c97fcd] cannot be updated because it is not stopped"

1) If it is true, how can i make the invokeHTTP processor to clear the state when the updateAttribute processor is running

2) If it is not true, what are the step or setting i should set to make sure the invokeHTTP processor clear the state of the updateAttribute processor while both processor is running ?

thanks.

Re: Reset the Filename sequence no./counter everyday

techNerd — Sat, 14 Oct 2023 14:24:44 GMT

Hi @MattWho,

Any help that i can get from you? thanks. I would like to clear state everyday without stopping the processor. Thanks.

Re: Reset the Filename sequence no./counter everyday

MattWho — Mon, 16 Oct 2023 21:22:13 GMT

@techNerd
Clearing a processor components state requires stopping the processor before you can "clear the state". Stopped state is required because the the processor may be writing or updating state when you attempt to clear state which would cause issues. When stopped there is no need to worry about a race condition between writes and deletes.

That being said, reseting the sequence number stored in state to 0 can be accomplished using the advanced UI of the UpdateAttribute processor and a special reset-seq flowfile you feed into the processor at 00:00 each day. The advance UI of the UpdateAttribute processor works like if,the,else logic.

So you would set up a Rule "reset" and a condition (if), If condition is true the "Actions" are applied. If no Rules's conditions are true, the processor's non advanced UI properties are applied.
UpdateAttribute properties (same as you already have):

Click on "advanced" in lower left corner of processor configuration UI to open and configure Rules:

Now all you need to do is setup a GenerateFlowFile processor that feeds a FlowFile into the updateAttribute processor once a day to reset seq to 0 stored in tat UpdateAttributes processor's local state.

Optionally you could add a RouteOnAttribute processor after the UpdateAttribute to route out the sequence file for termination so it does not continue through your dataflow.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt