Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

NiFi: how to fulfill flow with massive of dates in attribute

Explorer

I want to make several invokeHTTP requests with dates from a date range in its URL.

So I have first date, for example 2020-05-01, and end date - 2020-05-31.

How automatically create massive: 2020-05-01, 2020-05-02, ..., 2020-05-31 and put it into URL.

Thanks in advance!

 

1 ACCEPTED SOLUTION

Super Collaborator

Hi @Maria_pl , generally speaking the approach is as follows:

1. Generate a dummy flow file that will trigger (GenerateFlowFile processor)

2. Next step is UpdateAttribute processor that sets the start date and end date as attributes in the flow file

3. ExecuteScript is next. This can be a python script, or whichever language you prefer, that will use the start and end attributes to list out all the dates in between.

4. If your script produces single file output of dates, you can then use SplitText processor to cut each row into its own flow file and from there each file will have its own unique date in your range.

 

Hope that makes sense. 

View solution in original post

1 REPLY 1

Super Collaborator

Hi @Maria_pl , generally speaking the approach is as follows:

1. Generate a dummy flow file that will trigger (GenerateFlowFile processor)

2. Next step is UpdateAttribute processor that sets the start date and end date as attributes in the flow file

3. ExecuteScript is next. This can be a python script, or whichever language you prefer, that will use the start and end attributes to list out all the dates in between.

4. If your script produces single file output of dates, you can then use SplitText processor to cut each row into its own flow file and from there each file will have its own unique date in your range.

 

Hope that makes sense. 

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.