Setting TTL per key when writing to redis

[#1]

Hello community

I was wondering if anyone has any experience with writing entries into redis with dynamic TTLs. From what I can tell, it is only possible to set the TTL via the DistributedMapCacheClientService and as such it is set 'globally' per flow. For me the ideal case would've been if it was possible to set the TTL based on flowfile attributes.

Any advice would be greatly appreciated. Thanks 🙂

[#2 · Accepted solution]

@Green_ 

The MapCacheClientService does not provide any TTL capability. It simply communicates with the MapCacheServer. The MapCacheServer has some eviction strategies available, but none are based on some TTL attribute. 

The RedisDistributedMapCacheClientService controller service can be configured with a TTL; however, it can not be set dynamically via a FlowFile Attribute since it does not support NiFi Expression Language.  

This would require an improvement to this controller service.  I'd recommend creating a Apache NiFi Jira (https://issues.apache.org/jira/projects/NIFI/issues) requesting such an improvement with your use case.  Ideally this would involve enabling support for NiFi Expression language so that each FlowFile could pass a unique TTL.  Would also need to handle scenario when a FlowFile is missing the TTL attribute:
- Use some default TTL.
- Have separate "default TTL" configurable property that is used when Attribute is not set on FlowFile.
- Have FlowFile route to failure when attribute not set (this might not be possible).

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt