Support Questions
Find answers, ask questions, and share your expertise

Nifi Hangs on ExtractGrok after a while

New Contributor

Hi All,

I am extracting logs that needs to get parsed so I use ExtractGrok. It works for a while then just hangs after processing about 1500 files. If I stop (and terminate) the threads it starts working again but eventually hangs. This seems to only happen with the ExtractGrok processor..

The nifi dump reports ...

"Timer-Driven Process Thread-9" Id=196 TIMED_WAITING  on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6f9ed8d2
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(
        at java.util.concurrent.LinkedBlockingQueue.poll(
        at org.apache.nifi.processors.standard.HandleHttpRequest.onTrigger(
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(
        at org.apache.nifi.controller.tasks.ConnectableTask.invoke(
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$
        at org.apache.nifi.engine.FlowEngine$
        at java.util.concurrent.Executors$
        at java.util.concurrent.FutureTask.runAndReset(
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(
        at java.util.concurrent.ScheduledThreadPoolExecutor$
        at java.util.concurrent.ThreadPoolExecutor.runWorker(
        at java.util.concurrent.ThreadPoolExecutor$
        Number of Locked Synchronizers: 1
        - java.util.concurrent.ThreadPoolExecutor$Worker@669603c5

How could I solve this? I am not sure why this would be going on

I have Nifi running on a cluster of 4 nodes with 4 cpu and 16 gigs of memory. I expect it to take some time to run but constantly hanging after a few thousand files?

Many thanks,