Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to increment page Number of Fetch data in Apache NIFI

How to increment page Number of Fetch data in Apache NIFI

Explorer

Hello,

I have Fetch Open source API data using GetHTTP processor and save as text file.

Is there any way to auto increment page because size of data is more.

Thanks

1 REPLY 1
Highlighted

Re: How to increment page Number of Fetch data in Apache NIFI

Super Guru
@Mitthu Wagh

You can use UpdateAttribute Processor by storing the state and incrementing the last state value then pass the attribute value to get HTTP response.

Flow:

As GetHTTP processor doesn't accept any incoming connections use InvokeHTTP processor instead of GetHTTP processor.

87450-flow.png

UpdateAttribute Configs:

87451-ua.png

In this processor i'm using getStateValue function to get the value for the variable and adding one to it so output flowfile from UpdateAttribute will have seq attribute with 1 as value in your first run.

Then use the attribute in your InvokeHTTP processor to increment the page number.

Refer to this link if you want to reset the state once the number reaches to limit.

-

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

Don't have an account?
Coming from Hortonworks? Activate your account here