Skip to main content
Maria_pl
Maria_pl
Silver ⭐️
June 12, 2020
Solved

NiFi: how to fulfill flow with massive of dates in attribute

I want to make several invokeHTTP requests with dates from a date range in its URL.

So I have first date, for example 2020-05-01, and end date - 2020-05-31.

How automatically create massive: 2020-05-01, 2020-05-02, ..., 2020-05-31 and put it into URL.

Thanks in advance!

 

aakulov
aakulovAnswer
Platinum ⭐️
June 12, 2020

Hi @Maria_pl , generally speaking the approach is as follows:

1. Generate a dummy flow file that will trigger (GenerateFlowFile processor)

2. Next step is UpdateAttribute processor that sets the start date and end date as attributes in the flow file

3. ExecuteScript is next. This can be a python script, or whichever language you prefer, that will use the start and end attributes to list out all the dates in between.

4. If your script produces single file output of dates, you can then use SplitText processor to cut each row into its own flow file and from there each file will have its own unique date in your range.

 

Hope that makes sense. 

View in thread

1 reply

aakulov
Platinum ⭐️
aakulovAnswer
Platinum ⭐️
June 12, 2020

Hi @Maria_pl , generally speaking the approach is as follows:

1. Generate a dummy flow file that will trigger (GenerateFlowFile processor)

2. Next step is UpdateAttribute processor that sets the start date and end date as attributes in the flow file

3. ExecuteScript is next. This can be a python script, or whichever language you prefer, that will use the start and end attributes to list out all the dates in between.

4. If your script produces single file output of dates, you can then use SplitText processor to cut each row into its own flow file and from there each file will have its own unique date in your range.

 

Hope that makes sense. 

Translate
Terms & ConditionsAccessibility statement