@apk132  What is the full exception you are seeing so maybe i can dig deeper as to why that specific exception is not routing to failure? Matt ... View more

@apk132  I see in your latest image that the PublishKafka_2_6 has successfully completed 21 tasks/thread in the last 5 minutes. I also see a red bulletin on the processor.  What is the exception being seen?  It looks like it may not be a hung thread? and perhaps a session rollback is happening instead of routing to failure. That may be how some types of exceptions are handled by this processor even though you have configured the Failure Strategy as "Route to Failure" for "Route On Failure".   Matt ... View more

@samrathal  Is your Windows server hosting a SFTP server that you can connect to?  If not, I'd expect GetSFTP/FetchSFTP to fail to connect.   If you can't reach your Windows file from command line on the server where NiFi resides, you are not going to be able to reach via a NiFi dataflow. 1. Setting up a windows share that you mount to your linux box would allow you to use ListFile and FetchFile processors. 2. Setting up an SFTP server on your Windows server would allow you to use ListSFTP and FetchSFTP processors. 3. Setting up SMB on your Windows server would allow you to use the ListSMB and FetchSMB processors. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@mslnrd  Would not expect to see any error in the NiFi logs. The TLS exchange happens at a lower level and it is expected that if server (NiFi) is configured to REQUIRE a client certificate and one is not provided, the connection is closed.   The requirements for the certificate used by NiFi-Registry are no different then NiFi.  The complete Certificate Authority (CA) chain for the client auth certificate in the NiFi-Registry must be present in NiFi's truststore.   Similarly the the complete Certificate Authority (CA) chain for the client auth certificate in the NiFi must be present in NiFi-Registry's truststore.   This is only way that the mutual TLS exchange/handshake will be successful negotiated. While you can use the NiFi TLS toolkit to generate your certificates, that is not a requirement.  You can use free services or generate your own certificates as long as the meet the requirements. keystore must contain only one PrivateKeyEntry The PrivateKeyEntry must have both ClientAuth and ServerAuth ExtendedKeyUsage (EKU)s. The PrivateKeyEntry must include a Subject Alternative Name (SAN) entry matching the hostname of the server on which that certificate will be used for serverAuth.  You can have multiple SANs as your server may be known by a hostname, IP, or secondary hostname. The truststore will contain one too many TrustedCertEntries (public certificates).   Each TrustedCertEntry is for either an intermediate CA(s) or root CA(s).  An intermediate CA would be any certificate authority where the owner is different than the issuer/signer of that public certificate.  Root CAs will have same owner and issuer.   A complete trustchain means you have every CA leading from the issuer signer of your client certificate to the root CA.  There may be 0 or more intermediate CAs between the client and rootCA. The only way the NiFi and NiFi-Registry will authenticate with each other is through a mutual TLS exchange.  The NiFi certificate DN(s) will need to be authorized within your NiFi-Registry and so will your NiFi user.  The NiFi host(s) will proxy requests to NiFi-Registry on  behalf of the authenticated user in NIFi.  Even though the NiFi is acting as a proxy, the user will still need be properly authorized as the results form the proxy request will be for only the buckets and flows for which the proxied user is authorized against. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@apk132  What the first image you shared tells me is that your PublishKafka_2_6 processor received 3 FlowFile on its inbound connection.  Two of those were processed (at least one of those two resulted in an error triggering the red bulletin in upper right corner of the processor).  The third FlowFile is still associated to a running thread on the processor (indicated by the small "1" in the upper right corner or the processor) The stats show that 48 tasks successfully completed in the past 5 minutes yet you are saying this one FlowFile has been stuck that entire time?  How has the processor been scheduled? Is this a multi-node NiFi cluster?  Are the FlowFiles that are successfully processed being processed on a different node than the hung FlowFile is queued? When it comes to what appears to be a hung thread, getting a series of thread dumps of NIFi JVM spaced apart by 1 min or more can be useful for analysis.  You'll be looking to see if this thread stack is progressing or always showing exact same stack.  If it is changing then thread is not hung, but rather just long running.  If it is not changing, closure examination of the thread stack would help determine what that thread is waiting on.   If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@mslnrd  The NiFi Registry configuration property: nifi.registry.security.needClientAuth= is used to control whether NiFi-registry whether the TLS exchange is mutual (2-way TLS handshake if set to true) or 1-way TLS handshake (when set to false). At a very high level, in a mutual TLS handshake both the server and the client need to provide their certificate in the the TLS handshakes.  When you changed this to false, NiFi-Registry no longer "required" that a client certificate is presented to identify the client who connected to the NiFi-Registry. If no other method of authentication is enabled, NiFi-Registry will treat the client as "anonymous".   NiFi-Registry supports anonymous user access to the NiFi-Registry UI.  This access is limited to accessing already created "public" buckets and ability to read flows version controlled in to a "public" bucket. The ability to create buckets, administer buckets and flows, and write version controlled flows to any bucket would require an authenticated and authorized user.   So it is not clear how this simple change solved your issue.  I think this change simple allowed the anonymous access only. You'll need to inspect the keystore and truststore used by both NiFi (registry client ssl context) and NiFi-Registry (configured in nifi-registry.properties file) to make sure a mutual TLS exchange is possible.   If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@sarithe Mainly seen with ReplavceTEXT?  This processor reads and writes to the content repository.  How is Disk I/O for your content_repository?  How many concurrent tasks configured on this processor? How large is the content that the replaceText is evaluating?  How is this processor configured? This processor will also read all or part of that content (depending on configuration) in to heap memory in order to do the text replacement. Matt ... View more