Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi Wait/Notify flow not working as expected

Highlighted

NiFi Wait/Notify flow not working as expected

New Contributor

We're using a Wait/Notify pattern in one of our flows to overcome a race condition problem (object x is referencing object y, object x will wait until object y is created. Creation of object y will generate the notification).

This basically works for most of our flow files. Files are waiting in queue and are released when notified as expected. However, for some flow files the notify- signal for object y is sent but object x will still be stuck in waiting state, not reacting on it.

I have enabled persistence on the cache and verified that the notify- signals are really added to it. I have juggled the different parameters of the wait/notify- processors without any success (it should require minimal configuration though since we only have a 1 to 1 relation between wait and notify signals).

Any ideas?

We're using NiFi 1.6.0.

2 REPLIES 2

Re: NiFi Wait/Notify flow not working as expected

New Contributor

I created a script to get all the keys (using DistributedCacheMapService) from the cache and it seems that the keys we're waiting for are actually not in it. However, looking at the snapshot file for the cache on disk it seems that the keys are there.

Not sure about the life cycle for cached entries. When are they in the journal file and when are they in the snapshot file?

Re: NiFi Wait/Notify flow not working as expected

New Contributor

Another observation. We're running a small cluster with two NiFi instances. Depending on which instance that runs I get different results from the cache extraction explained above (the DistributedCacheMapService points at cache at localhost). Is the cache bound to each NiFi instance? I expected it to be distributed?

Don't have an account?
Coming from Hortonworks? Activate your account here