question Re: problem receiving Data on HandleHTTPRequest not working properly in Support Questions

problem receiving Data on HandleHTTPRequest not working properly

AndyAjram — Tue, 19 Nov 2019 23:18:19 GMT

I have a problem with the HandleHTTPRequest Processor, I'm using it to receive Data from Sensors via a path and a port, the problem is that even If sensors send Data and when using wireshark I see Data coming from sensors to my Integration server in the port assigned , but the processor doesn't show anything until I restart NIFI , Data reach the processor for some minutes or sometime for 2 days and after it stop again , don't know what is the problem , because restarting Nifi every time is not a good solution for me , someone can help me on that ?

Re: problem receiving Data on HandleHTTPRequest not working properly

stevenmatison — Tue, 19 Nov 2019 16:23:13 GMT

It sounds like your NiFi needs some tweaking, to get things stable. You should not have to restart NiFi. If it is stopping/crashing/clogging up after some time online running as expected, this is usually due to instability of NiFi itself, not the processor.

Can you provide some details about the spec/sizing for the nifi? Also what is the size of data arriving?

Re: problem receiving Data on HandleHTTPRequest not working properly

AndyAjram — Tue, 19 Nov 2019 16:28:02 GMT

this is what comes today after restarting Nifi , and stop after :

what do you mean by NIFI size ?

Re: problem receiving Data on HandleHTTPRequest not working properly

stevenmatison — Tue, 19 Nov 2019 17:28:39 GMT

Sizing and spec of the nifi node:

Node Ram and Cores

Min/max memory for nifi

Is the node dedicated to NiFi? If the Node has other duties too, these may prevent NiFi from having enough resources causing it to crash or get locked up during long term testing (days/weeks). Restarting NiFi resets this condition, and then it works again.

First step is to increase the min/max memory, but sharing the Nifi Node Environment details will help us provide more specific details.

Re: problem receiving Data on HandleHTTPRequest not working properly

AndyAjram — Tue, 19 Nov 2019 17:45:17 GMT

The RAM of Nifi Node is 32 G and it has a disk space dedicated just to NIFI and its logs with 250 G , I delete logs every time to not face the problem of low disk , I have just NIFI running in this Node and some small applications like postman and Robot3 not very big to prevent him doing his Job

Re: problem receiving Data on HandleHTTPRequest not working properly

stevenmatison — Tue, 19 Nov 2019 18:02:52 GMT

Ok so bump up the min/max ram in the nifi.properties or in the config (if using ambari or other). Start with 2gb / 4gb, and go up in steps until you get stability you are looking for. You should not have to go as high as 16gb / 32gb to be stable. The default is very low.

Re: problem receiving Data on HandleHTTPRequest not working properly

AndyAjram — Tue, 19 Nov 2019 18:07:04 GMT

Okey I will try this tomorrow and give you feedback , thank you so much

Re: problem receiving Data on HandleHTTPRequest not working properly

AndyAjram — Wed, 20 Nov 2019 08:36:28 GMT

Hi,

I tried the solution of increasing the Memory , but nothing has changed , the processor still not working

Re: problem receiving Data on HandleHTTPRequest not working properly

AndyAjram — Wed, 27 Nov 2019 08:00:34 GMT

I solved the issue with increasing the Request expiration which was set to 1 min .