Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi: how to fulfill flow with massive of dates in attribute

Solved Go to solution
Highlighted

NiFi: how to fulfill flow with massive of dates in attribute

Explorer

I want to make several invokeHTTP requests with dates from a date range in its URL.

So I have first date, for example 2020-05-01, and end date - 2020-05-31.

How automatically create massive: 2020-05-01, 2020-05-02, ..., 2020-05-31 and put it into URL.

Thanks in advance!

 

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NiFi: how to fulfill flow with massive of dates in attribute

Rising Star

Hi @Maria_pl , generally speaking the approach is as follows:

1. Generate a dummy flow file that will trigger (GenerateFlowFile processor)

2. Next step is UpdateAttribute processor that sets the start date and end date as attributes in the flow file

3. ExecuteScript is next. This can be a python script, or whichever language you prefer, that will use the start and end attributes to list out all the dates in between.

4. If your script produces single file output of dates, you can then use SplitText processor to cut each row into its own flow file and from there each file will have its own unique date in your range.

 

Hope that makes sense. 

View solution in original post

1 REPLY 1
Highlighted

Re: NiFi: how to fulfill flow with massive of dates in attribute

Rising Star

Hi @Maria_pl , generally speaking the approach is as follows:

1. Generate a dummy flow file that will trigger (GenerateFlowFile processor)

2. Next step is UpdateAttribute processor that sets the start date and end date as attributes in the flow file

3. ExecuteScript is next. This can be a python script, or whichever language you prefer, that will use the start and end attributes to list out all the dates in between.

4. If your script produces single file output of dates, you can then use SplitText processor to cut each row into its own flow file and from there each file will have its own unique date in your range.

 

Hope that makes sense. 

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here