Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to increase number of requests In Invoke HTTP processer

avatar
Explorer

Hi,

I am using InvokeHTTP  processer  in Apache Nifi.  Only 20 requests are getting POST at a time. how can I increase the number of requests post through InvokeHttp.

Do we have any parameter we can add or change in the InvokeHTTP processer it self orDo we need to change any config level changes.
Thanks In advance.

 

1 ACCEPTED SOLUTION

avatar
New Contributor

@udayAleGMSocrates wrote:

Hi,

I am using InvokeHTTP  processer  in Apache Nifi.  Only 20 requests are getting POST at a time. how can I increase the number of requests post through InvokeHttp.

Do we have any parameter we can add or change in the InvokeHTTP processer it self orDo we need to change any config level changes.
Thanks In advance.

 


Hello, @udayAle 

To increase the number of requests that can be sent concurrently through the InvokeHTTP processor in Apache NiFi, you can adjust the following settings:

  • Concurrent Tasks: Increase the number of concurrent tasks for the InvokeHTTP processor. This can be done by:
    Right-clicking on the processor.
    Selecting ‘Configure’.
    Going to the ‘Scheduling’ tab.
    Adjusting the ‘Concurrent tasks’ setting to a higher number.
  • Run Schedule: Decrease the run schedule to allow the processor to execute more frequently. This is also in the ‘Scheduling’ tab of the processor’s configuration.
  • FlowFile Batching: If your flow allows, you can use processors like MergeContent to batch multiple FlowFiles into a single request, reducing the number of HTTP calls made.
  • System Resource Considerations: Ensure that your NiFi instance has enough system resources (CPU, memory) to handle the increased load.

Please note that increasing the number of concurrent requests may put additional load on both NiFi and the target server, so it’s important to monitor the performance and stability of your system after making these changes. If you’re unsure about the optimal settings, it’s a good practice to incrementally increase the concurrent tasks and observe the impact.

If you need further assistance or have more questions, feel free to ask!.

Best Regard,
ryan1969

Best Regard,
ryan1969

View solution in original post

3 REPLIES 3

avatar
New Contributor

@udayAleGMSocrates wrote:

Hi,

I am using InvokeHTTP  processer  in Apache Nifi.  Only 20 requests are getting POST at a time. how can I increase the number of requests post through InvokeHttp.

Do we have any parameter we can add or change in the InvokeHTTP processer it self orDo we need to change any config level changes.
Thanks In advance.

 


Hello, @udayAle 

To increase the number of requests that can be sent concurrently through the InvokeHTTP processor in Apache NiFi, you can adjust the following settings:

  • Concurrent Tasks: Increase the number of concurrent tasks for the InvokeHTTP processor. This can be done by:
    Right-clicking on the processor.
    Selecting ‘Configure’.
    Going to the ‘Scheduling’ tab.
    Adjusting the ‘Concurrent tasks’ setting to a higher number.
  • Run Schedule: Decrease the run schedule to allow the processor to execute more frequently. This is also in the ‘Scheduling’ tab of the processor’s configuration.
  • FlowFile Batching: If your flow allows, you can use processors like MergeContent to batch multiple FlowFiles into a single request, reducing the number of HTTP calls made.
  • System Resource Considerations: Ensure that your NiFi instance has enough system resources (CPU, memory) to handle the increased load.

Please note that increasing the number of concurrent requests may put additional load on both NiFi and the target server, so it’s important to monitor the performance and stability of your system after making these changes. If you’re unsure about the optimal settings, it’s a good practice to incrementally increase the concurrent tasks and observe the impact.

If you need further assistance or have more questions, feel free to ask!.

Best Regard,
ryan1969

Best Regard,
ryan1969

avatar
Explorer

Thanks for the above solution. Can you please help me on below query.

how to configure email alerts in Nifi whenever files stuck in queue for more than 3 minutes

avatar
Master Mentor

@udayAle 
Please start a new community question for your unrelated follow-up question above.  Responses to an unrelated question will lead to confusion to other community members who may be having similar problems.

You can use @<username>  to notify specific people about new community questions.

Thank you,
Matt