Apache nifi remote Input Port connection hang issue

[#1]

Setup: We have 2 instances of nifi communicating with each other over RemoteProcessingGroup(nifi-1)-->remoteInput_port(nifi-2) connection

Issue: Sometimes the remote Input port (in nifi 2) goes into hang state and stops accepting flowfiles from nifi-1.

Description: We are sending flowfiles from nifi1 to nifi2 to transfer some varioable for further processing. These flowfiles don't contain any content but only attributes(5 to 10) and hence they are in bytes. Sometimes the remote Input port (in nifi 2) goes into hang state. It simply shows that it is in running state ans 1 thread is constantly active. Once we stop it, it doesn't get active again and stays in stop state even if we try to start it. It basically hangs and doesn't even disables or delete. To mitigate this issue, we simply have to create a new remote Input port and route to this new remote port for ongoing flow. But this issues blocks our pipeline till the time it is manually intervened. Also, it is very difficult to reproduce this issue as it randomly happens.

Help: what could be the reason for this issue and has anyone faced it? Is there any way to avoid this issue in future as this issue blocks our production flow?

[#2]

@sundaram_idfc 

Additional information may help here:

1. What versions of NiFi are bing used by each NiFi instance you have installed?
2. What Java version is being used by each of these NiFi instances?
3. What is your configured "Max Timer Driven thread count" setting under global menu --> controller settings?
4. How many cores does your server have that is running NiFi?
5. Any chance you are occasionally sending a larger FlowFile?
6. When in this state, did you collect a series of NiFi thread dumps to see if the thread is progressing or what it is waiting or blocked by (nifi.sh dump <dump filename>)
7. Any WARN or ERROR related log entries in the nifi-app.log related to this port, OutOfMemory, or file handles?

Thanks,

Matt