nifi : Java heap space

[#1]

hi all

how configure java heap memory to avoid this errors ?

It seems failed on SplitText processor.

thanks

nifi-app_2016-12-26_16.0.log:2016-12-26 16:22:46,484 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:23:03,484 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:23:03,492 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:24:51,066 WARN [Cluster Socket Listener] org.apache.nifi.io.socket.SocketListener Socket Listener encountered exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:Caused by: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:Caused by: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:30:14,746 ERROR [Timer-Driven Process Thread-1] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:30:21,387 ERROR [Timer-Driven Process Thread-1] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:30:21,412 WARN [Timer-Driven Process Thread-1] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:31:24,862 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:31:25,682 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:31:25,684 WARN [Timer-Driven Process Thread-2] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:32:24,871 ERROR [Timer-Driven Process Thread-6] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:32:25,933 ERROR [Timer-Driven Process Thread-6] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:32:25,936 WARN [Timer-Driven Process Thread-6] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:33:23,298 ERROR [Timer-Driven Process Thread-4] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:33:24,231 ERROR [Timer-Driven Process Thread-4] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:33:24,234 WARN [Timer-Driven Process Thread-4] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:38:39,403 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:38:39,407 ERROR [FileSystemRepository Workers Thread-2] o.a.n.c.repository.FileSystemRepository Failed to handle destructable claims due to java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:38:39,410 ERROR [Listen to Bootstrap] org.apache.nifi.BootstrapListener Failed to process request from Bootstrap due to java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:38:39,411 ERROR [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for container default due to java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:Caused by: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:Caused by: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:Caused by: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:Caused by: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:38:40,578 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:38:40,580 WARN [Timer-Driven Process Thread-2] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:45:09,764 ERROR [Timer-Driven Process Thread-1] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:45:09,800 WARN [Cluster Socket Listener] org.apache.nifi.io.socket.SocketListener Socket Listener encountered exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:45:10,982 ERROR [Timer-Driven Process Thread-1] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:45:10,991 WARN [Timer-Driven Process Thread-1] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:50:00,311 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:50:01,617 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:50:01,622 WARN [Timer-Driven Process Thread-2] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:Caused by: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:51:59,363 ERROR [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Failed to cleanup archive for container default due to java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:51:59,363 WARN [Cluster Socket Listener] org.apache.nifi.io.socket.SocketListener Socket Listener encountered exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:53:54,723 ERROR [Timer-Driven Process Thread-4] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:53:54,723 ERROR [Listen to Bootstrap] org.apache.nifi.BootstrapListener Failed to process request from Bootstrap due to java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:53:54,726 ERROR [Provenance Maintenance Thread-3] o.a.n.p.PersistentProvenanceRepository Failed to expire Provenance Query Results due to java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:53:55,430 ERROR [Timer-Driven Process Thread-4] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:53:55,431 WARN [Timer-Driven Process Thread-4] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:54:50,287 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:54:50,775 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:54:50,776 WARN [Timer-Driven Process Thread-8] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:56:05,576 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:56:05,584 ERROR [Timer-Driven Process Thread-7] o.a.nifi.processors.standard.TailFile TailFile[id=54d23958-bb03-1f9b-acf1-6ef0d4034421] TailFile[id=54d23958-bb03-1f9b-acf1-6ef0d4034421] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:56:05,584 ERROR [Timer-Driven Process Thread-7] o.a.nifi.processors.standard.TailFile TailFile[id=54d23958-bb03-1f9b-acf1-6ef0d4034421] TailFile[id=54d23958-bb03-1f9b-acf1-6ef0d4034421] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:56:05,584 WARN [Timer-Driven Process Thread-7] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding TailFile[id=54d23958-bb03-1f9b-acf1-6ef0d4034421] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:56:05,944 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:56:05,945 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:59:42,722 ERROR [Timer-Driven Process Thread-3] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process due to java.lang.OutOfMemoryError: Java heap space; rolling back session: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:59:44,490 ERROR [Timer-Driven Process Thread-3] o.a.nifi.processors.standard.SplitText SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:2016-12-26 16:59:44,491 WARN [Timer-Driven Process Thread-3] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding SplitText[id=77273814-e6ed-1596-bac6-55c0410b05a9] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
nifi-app_2016-12-26_16.0.log:java.lang.OutOfMemoryError: Java heap space

[#2 · Accepted solution]

I use this article

https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.h...

A ton while configuring nifi.

Heap size is set in the bootstrap.conf file. How to set it is in the article.

[#3]

@mayki wogno please advise if your question has been answered.

[#4]

https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.h...

https://community.hortonworks.com/questions/46867/javalangoutofmemoryerror-permgen-space-in-nifi.htm...

[#5]

I ran into the same issue of OOM on NiFi and discovered that the Init and Max memory allocation were set to 512M. Setting both to 4GB resolved my issue. Thanks @Sunile Manjee for recommending the article by @Matt Clarke.

[#6]

@mayki wogno

Every FlowFile that exists consists of two parts, FlowFile content and FlowFile Attributes. While the FlowFile's content lives on disk in the content repository, NiFi holds the "majority" of the FlowFile attribute data in the configured JVM heap memory space. I say "majority" because NiFi does swapping of Attributes to disk on any queue that contains over 20,000 FlowFiles (default, but can be changed in the nifi.properties).

So some common reason for running out of heap memory include:

1. High volume dataflow with lots of FlowFiles active any any given time across your dataflow. (Increase configured nifi heap size in bootstrap.conf to resolve)

2. Creating a large number of Attributes on every FlowFile. More Attributes equals more heap usage per FlowFile. (Increase configured nifi heap size in bootstrap.conf to resolve and/or reduce the configured swap threshold)

3. Writing large values to FlowFile Attributes. Extracting large amounts of content and writing it to an attribute on a FlowFile will result in high heap usage. Try to avoid creating large attributes when possible. (Increase configured nifi heap size in bootstrap.conf to resolve and/or reduce the configured swap threshold)

4. Using the MergeContent processor to merge a very large number of FlowFiles. NiFi can not merge FlowFiles that are swapped, so all these FlowFile's attributes must be in heap when the merge occurs. If merging a very large number of FlowFiles is needed, try using two MergeContent processors in series with one another. Have first merge a max of 10,000 FlowFiles and the second then merge those 20,000 FlowFile files in to even larger bundles. (Increase configured nifi heap size in bootstrap.conf also help)

5. Using the SplitText processor to split one File in to a very large number of FlowFiles. Swapping of a large connection queue will not occur until after the queue has exceeded swapping threshold. The SplitTEXT processor will create all the split FiLowFiles before committing them to the success relationship. Most commonly seen when SpitText is used to split a large incoming FlowFile by every line. It is possible to run out of heap memory before all the splits can be created. Try using two SplitText processors in series. Have the first split the incoming FlowFiles in to large chunks and the second split them down even further. (Increase configured nifi heap size in bootstrap.conf also help)

Thanks,

Matt

[#7]

Thinks for all your suggestion.  By increasing HEAP size issue resloved.

Thanks,

Suryakant