Created 10-11-2023 06:40 AM
Hi,
Every 5 min i have a file generated from the NiFi Processor GenerateFlowFile:
filename_0.xml
filename_1.xml
filename_2.xml
....
I would like to asked how do i reset the sequence no. everyday ?
Thanks. Would appreciate help with example and explanation.
The first method i found is using nextInt():
the second method is using seq:
Please advise me with steps and example. thanks
Created on 10-11-2023 10:52 AM - edited 10-11-2023 10:53 AM
Maybe you could reset the state via NiFi-REST-API at beginning of your flow or separately on a schedule cron every morning
Could be:
POST "https://[ip:port]/nifi-api/processors/${processor-id}/state/clear-requests"
This is the request which NiFi itself uses when you go to the processor in the UI and choose the menu-option "view state" -> "clear state".
Created 10-11-2023 04:42 PM
Hi @drewski7,
Thanks for the replying.
From your reply, I have added in NiFi InvokeHTTP processor which will automatically help to reset my seq no. counter every morning. I am using the second method above which using seq no. which store state locally.
I have set the HTTP Method to POST and HTTP URL to https://[ip:port]/nifi-api/processors/${processor-id}/state/clear-requests
I have scheduled the NiFi InvokeHTTP Processor to run every morning 00:00 using CRON driven Scheduling Strategy
Below image are the NiFi UpdateAttribute Processor which show the processor ID that i used inside my HTTP URL
Just to check did i did it correctly as what you mention in your reply ?
I tried to change my computer time to 11:55 pm and let it run to 00:00 the next day, but it does not invokeHTTP POST request once it reach 00:00. I might have made some mistake somewhere.
appreciate if you could advise me with example and explanation. Thanks a lot.
Created on 10-12-2023 08:04 AM - edited 10-12-2023 08:55 AM
Hi @drewski7 ,
I noticed that the invokeHTTP processor will only POST request to clear state when the updateAttribute processor which contain the state is STOP.
Error Message:
i got from invokeHTTP processor "UpdateAttribute[id=e6c537bd-c24b-1801-6fdd-b37c23c97fcd] cannot be updated because it is not stopped"
1) If it is true, how can i make the invokeHTTP processor to clear the state when the updateAttribute processor is running
2) If it is not true, what are the step or setting i should set to make sure the invokeHTTP processor clear the state of the updateAttribute processor while both processor is running ?
thanks.
Created 10-14-2023 07:24 AM
Hi @MattWho,
Any help that i can get from you? thanks. I would like to clear state everyday without stopping the processor. Thanks.
Created on 10-16-2023 02:19 PM - edited 10-16-2023 02:22 PM
@techNerd
Clearing a processor components state requires stopping the processor before you can "clear the state". Stopped state is required because the the processor may be writing or updating state when you attempt to clear state which would cause issues. When stopped there is no need to worry about a race condition between writes and deletes.
That being said, reseting the sequence number stored in state to 0 can be accomplished using the advanced UI of the UpdateAttribute processor and a special reset-seq flowfile you feed into the processor at 00:00 each day. The advance UI of the UpdateAttribute processor works like if,the,else logic.
So you would set up a Rule "reset" and a condition (if), If condition is true the "Actions" are applied. If no Rules's conditions are true, the processor's non advanced UI properties are applied.
UpdateAttribute properties (same as you already have):
Click on "advanced" in lower left corner of processor configuration UI to open and configure Rules:
Now all you need to do is setup a GenerateFlowFile processor that feeds a FlowFile into the updateAttribute processor once a day to reset seq to 0 stored in tat UpdateAttributes processor's local state.
Optionally you could add a RouteOnAttribute processor after the UpdateAttribute to route out the sequence file for termination so it does not continue through your dataflow.
If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt