@abdullahvvs  What exceptions are you seeing in the nifi-app.log when this processor gets scheduled to execute? Does your NiFi service user have ability to execute python3? Does your NiFi service user have ability to travers the directory tree and read /opt/nifi/nifi-current/Testxx.py? If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt   ... View more

@sinRudra  I recommend reading through the comments, suggestions, and troubleshooting done within the following Apache NiFi Jira to see if you are encountering similar observations. https://issues.apache.org/jira/browse/NIFI-10143 If so, it may be best to create a new Apache JIra for your specific character set issues to see what the committer community can do for you. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@learner-loading  The Site-To-Site functionality in NiFi has not evolved a whole lot since it was first introduced.  You have an initial client server connection over http(s) followed by FlowFile transfer via either the same HTTP(s) connection or via a dedicated Raw socket connection depending on the configuration on the server side of the connection. To use a mutual TLS HTTPS connection for initial connection, the following property must be set tp true: nifi.remote.input.secure=true In order to set this to true, NiFi must be configured with a keystore and truststore in the nifi.properties file as well. To support HTTP(s) FlowFile transfer the following property must be set to "true": nifi.remote.input.http.enabled=true   To support RAW socket connection for FlowFile transfer, the following property must be configured with   a port not in use by all nodes in yoru NiFi cluster: nifi.remote.input.socket.port=10443   The Remote Process Group is always the "client" in the site-to-site connection. The Remote Input and Remote output ports are then the "server" side of the connection. So when the Remote Process Group (RPG) has been configured with a URL(s) (If target NiFi is a cluster setup, you can optionally provide a comma separated list of URLs for the hosts  in the same target cluster to offer some redundancy in case one of the target nodes is down), the RPG will start automatically connecting to first URL to try to fetch Site-To-Site (S2S) details about the server side of this connection.  Those server details include if the server side is configured secure or unsecured, if it supports HTTP, RAW, or both, the URLs for all connected nodes in target cluster, NiFi HTTTP port for all connected nodes, RAW port for all connected nodes, total amount of queued FlowFiles on each connected node, Available Remote Input and Output ports on target NiFi cluster. With an HTTPS enabled Site-To-Site, the keystore and truststore files configured on both source/client Nifi and target/server NiFi will be used in the mutual TLS handshake connection made (The client certificate DN is used as the client identity passed to the server that needs to be properly authorized on the target Remote Input and Output ports.  If authorization is successful and the ROPG has "enabled transmission" FlowFiles will be able to transfer from RPG to Remote Input Port or  From a Remote Output Port to the RPG.  Better load-balancing is achieved with RPG pushing to Remote Input Ports. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@Tenda  What processor become unresponsive? You mean that the processor indicates that it is currently executing a thread (small number shown in upper right corner); however, all the stats on the processor for in, out, tasks show 0 for last 5 minutes? If tasks show numbers updating, then tasks/threads are executing and completing. If tasks is showing 0 for last 5 minutes or very low for last 5 minutes and you see an active thread number in upper right corner of processor, it may be caused by a few reasons: Your CPU load average is how due to cpu intensive processors executing at same time. (would expect lag in UI if CPU was saturated) You have a processors configured with too many concurrent tasks leading to other processors not getting allocated a thread often enough. (If core load average is consistently low, you could increase the size of your max timer driven thread pool higher than 32. Java heap garbage collection (GC).  GC happens when your JVM heap usage reaches ~80% utilization. If your heap is too small, you could be experiencing lots of back to back GC.  All GC whether partial or full GC are stop-the-world events weaning JVM will do nothing while GC is happening.  If you heap is set to large, the GC stop-the-world may take much longer to complete. You have processors that have long running tasks or hung threads consuming threads from your available max timer driven thread pool thus limiting available threads for other components.  Only the examination of a series of multiple NiFi JVM thread dumps collected minutes apart will tell you if you have a long running task (thread dump shows change to thread indicating slow progress being made) or potentially hung thread (thread dumps all show same consistent output for thread.  When you have a processor that is in this state and "terminate" the thread on the processor, does the terminated thread (shown as small number with parenthesis "(1)") ever go away?  if not, that terminated thread never completed.  While "terminate" release FlowFile associated to that thread back to inbound connection queue and give user back full control of the processor.  The only way to "kill" a truly hung thread is by restarting the entire NiFi JVM.  Which you said you do once in awhile. Hope you find this information helps you drill deeper in to your issue and identify what is impacting you. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@tej_s  I recommend against switching to encrypted repositories while you still have content in your flowfile_repository, content_repository, and provenance_repository.   What version of Apache NiFi are you using? If you stop NiFi and delete the contents of these encrypted repositories and startup clean, do you encounter and issues like above with all new data being processed through your dataflows? If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more