@MarinaM Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  Thanks. ... View more

Thank you so much @MattWho for the detailed answer. The retry logic helped a lot, I have added 'RetryFlowFile' processors in between to avoid infinite loop of retry. ... View more

Thank you, @MattWho , for the additional information. I really appreciate you providing timely and clear solutions. You’re awesome! ... View more

@hus  There are two controller services you are using for your map cache: DistributedMapCacheServer - This controller service when started creates a separate map cache server on every node in a NiFi cluster. These map cache servers do not share cached entries between them. In Apache NiFi 2.x+ "Distributed" has been removed from their name to avoid confusion.  The "Max cache Entries" and "Eviction Strategy" control how cached entries are removed from the cache. DistributedMapCacheClientService - This Controller Service is used to write data to the specific Map cache server (server hostname). It also has "distributed" removed from its name as of Apache NiFi 2.x.   You are using the DetectDuplicate processor to interact with the above Controller services. While the DetectDuplicate processor has a configurable "Age Off Duration" setting,  ONLY cached entries where both the following conditions have been met will have the cache entry removed at that configured  age off: At least one duplicate has been detected. Age off duration has expired. So any cached entires for which a duplicate has not yet been detected, that entry will remain in the cache server until the "Max cache Entries" and "Eviction Strategy" settings result in the entry removal. So depending on what data you are caching, number set for "max cache Entries", and number of duplicates you detect, your cache server likely continues to grow to max and then eviction starts.  If you have a "Persistence Directory" configured, the cached data is also being written to that directory so that it is not lost in the event the NiFi instance or DistributedMapCache server is restarted.   This also means hat after a NiFi restart the persisted cache is loaded back into heap memory. Keep in mind that there are other external cache server options that do have HA, are distributed, and would not consume NiFi's heap or memory on the NiFi host if installed on a different server/host. RedisDistributedMapCacheClientService SimpleRedisDistributedMapCacheClientService HazelcastMapCacheClient CouchbaseMapCacheClient - Removed as of Apache NiFi 2.x HBase_2_ClientMapCacheService - Removed as of Apache NiFi 2.x CassandraDistributedMapCache - Removed as of Apache NiFi 2.x Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@pasq  Did you know that Cloudera makes managing and deploying your MiNiFi agents easy with Cloudera Edge Management?  Our ManagementHub provides a UI to build, deploy and update the dataflow you deploy to one too many MiNiFi agents. You can create parameters for various properties like passwords to provide those at deployment time through Edge Management Check out this Video. NiFi processors with "sensitive" properties are designed to encrypt those password when written to the flow.json.gz.  Even in MiNiFi these component classes will expect that the passwords are encrypted.  The defined sensitive.props.key property value in the minifi.properties or nifi.propertes is used in the encryption/decryption of these sensitive properties stored in the flow.json.gz file. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Scorpy257 As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. Thanks. ... View more

@NaveenSagar Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  Thanks. ... View more

@David26  I am not familiar with this specific processor "Plc4xListenRecordProcessor 0.12.0".   I could not find it in Apache NiFi 1.x or 2.x versions.  I could find it in Maven Central either.  Where did you download this nar that you added to your NiFi? What version of Apache NiFi are you using? You may need to reach out to the originator of this custom nar you have added to you NiFi for this processor specific query. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@BuffaloDaddy  What method of user authentication is configured in your NiFi? Single-user ldap-provider Kerberos-provider etc? Thank you, Matt ... View more