Support Questions
Find answers, ask questions, and share your expertise

NiFi: PutHiveStreaming Issue.

NiFi: PutHiveStreaming Issue.

Nifi not loading any thing in target Hive Managed Table (no partition, yes buckets) from Hive External Tables.

Nifi Processors:

  1. SelectHiveQL: Running successfully and sends out a Queue.
  2. PutHiveStreaming: Not able to pull Queue from SelectHiveQL processor. There is no error, it just keep running.

Already started NiFi.

Please advice.

Below is latest log:

=====================================================================

2018-08-14 12:27:23,215 INFO [main] org.eclipse.jetty.server.Server jetty-9.4.3.v20170317 2018-08-14 12:27:28,889 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=4651ms 2018-08-14 12:27:29,299 INFO [main] org.eclipse.jetty.server.session DefaultSessionIdManager workerName=node0 2018-08-14 12:27:29,299 INFO [main] org.eclipse.jetty.server.session No SessionScavenger set, using defaults 2018-08-14 12:27:29,301 INFO [main] org.eclipse.jetty.server.session Scavenging every 660000ms 2018-08-14 12:27:29,415 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@7d626f2{/nifi-jolt-transform-json-ui-1.5.0.3.1.2.0-7,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-jolt-transform-json-ui-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/extensions/nifi-standard-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-jolt-transform-json-ui-1.5.0.3.1.2.0-7.war} 2018-08-14 12:27:33,702 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=3744ms 2018-08-14 12:27:33,855 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@2c08c787{/nifi-update-attribute-ui-1.5.0.3.1.2.0-7,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-update-attribute-ui-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/extensions/nifi-update-attribute-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-update-attribute-ui-1.5.0.3.1.2.0-7.war} 2018-08-14 12:27:35,522 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=1264ms 2018-08-14 12:27:35,632 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@302da330{/nifi-standard-content-viewer-1.5.0.3.1.2.0-7,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-standard-content-viewer-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/extensions/nifi-standard-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-standard-content-viewer-1.5.0.3.1.2.0-7.war} 2018-08-14 12:27:35,727 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=63ms 2018-08-14 12:27:35,834 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@7c9372ed{/nifi-image-viewer-1.5.0.3.1.2.0-7,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-image-viewer-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/extensions/nifi-media-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-image-viewer-1.5.0.3.1.2.0-7.war} 2018-08-14 12:27:36,308 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=165ms 2018-08-14 12:27:36,333 INFO [main] org.eclipse.jetty.ContextHandler./nifi No Spring WebApplicationInitializer types detected on classpath 2018-08-14 12:27:36,467 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@4387d2f4{/nifi,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-web-ui-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-web-ui-1.5.0.3.1.2.0-7.war} 2018-08-14 12:27:36,816 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=175ms 2018-08-14 12:27:36,929 INFO [main] o.eclipse.jetty.ContextHandler./nifi-api No Spring WebApplicationInitializer types detected on classpath 2018-08-14 12:27:36,956 INFO [main] o.eclipse.jetty.ContextHandler./nifi-api Initializing Spring root WebApplicationContext 2018-08-14 12:27:44,383 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be '/root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/./conf/nifi.properties' 2018-08-14 12:27:44,391 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be '/root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/./conf/nifi.properties' 2018-08-14 12:27:44,392 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 145 properties from /root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/./conf/nifi.properties 2018-08-14 12:27:48,788 INFO [main] o.a.n.r.v.FileBasedVariableRegistry Loaded 116 properties from system properties and environment variables 2018-08-14 12:27:48,788 INFO [main] o.a.n.r.v.FileBasedVariableRegistry Loaded a total of 116 properties. Including precedence overrides effective accessible registry key size is 116 2018-08-14 12:27:48,980 INFO [main] o.a.n.c.r.WriteAheadFlowFileRepository Initialized FlowFile Repository using 256 partitions 2018-08-14 12:27:49,516 INFO [main] o.a.n.p.lucene.SimpleIndexManager Index Writer for ./provenance_repository/index-1534229719000 has been returned to Index Manager and is no longer in use. Closing Index Writer 2018-08-14 12:27:49,529 INFO [main] o.a.n.p.PersistentProvenanceRepository Recovered 21 records 2018-08-14 12:27:49,561 INFO [main] o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers for events starting with ID 21 2018-08-14 12:27:49,577 INFO [main] o.a.n.c.repository.FileSystemRepository Maximum Threshold for Container default set to 53599786106 bytes; if volume exceeds this size, archived data will be deleted until it no longer exceeds this size 2018-08-14 12:27:49,578 INFO [main] o.a.n.c.repository.FileSystemRepository Initializing FileSystemRepository with 'Always Sync' set to false 2018-08-14 12:27:49,931 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 finished recovering records. Performing Checkpoint to ensure proper state of Partitions before updates 2018-08-14 12:27:49,932 INFO [main] org.wali.MinimalLockingWriteAheadLog Successfully recovered 0 records in 14 milliseconds 2018-08-14 12:27:49,953 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 18 milliseconds (Stop-the-world time = 5 milliseconds, Clear Edit Logs time = 8 millis), max Transaction ID -1 2018-08-14 12:27:50,040 INFO [main] o.apache.nifi.controller.FlowController Not enabling RAW Socket Site-to-Site functionality because nifi.remote.input.socket.port is not set 2018-08-14 12:27:50,143 INFO [main] o.a.n.w.c.ApplicationStartupContextListener Starting Flow Controller... 2018-08-14 12:27:51,860 INFO [main] o.a.n.c.s.StandardControllerServiceProvider Created Controller Service of type org.apache.nifi.dbcp.hive.HiveConnectionPool with identifier 2ed06e6e-e34f-3b84-39ef-b4637ca2ebab 2018-08-14 12:27:51,881 INFO [main] o.a.nifi.groups.StandardProcessGroup HiveConnectionPool[id=2ed06e6e-e34f-3b84-39ef-b4637ca2ebab] added to StandardProcessGroup[identifier=3731b8ce-0165-1000-e94c-aaff03b543dd] 2018-08-14 12:27:53,580 INFO [main] o.a.n.c.s.StandardProcessScheduler Enabling HiveConnectionPool[id=2ed06e6e-e34f-3b84-39ef-b4637ca2ebab] 2018-08-14 12:27:57,256 INFO [main] o.a.nifi.groups.StandardProcessGroup Template[id=7a6ac69b-0ef8-4033-ad30-5ecf62507636] added to StandardProcessGroup[identifier=3731b8ce-0165-1000-e94c-aaff03b543dd] 2018-08-14 12:27:58,077 INFO [main] o.apache.nifi.controller.FlowController Successfully synchronized controller with proposed flow 2018-08-14 12:27:58,578 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 finished recovering records. Performing Checkpoint to ensure proper state of Partitions before updates 2018-08-14 12:27:58,578 INFO [main] org.wali.MinimalLockingWriteAheadLog Successfully recovered 1 records in 325 milliseconds 2018-08-14 12:27:58,927 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 1 Records and 0 Swap Files in 348 milliseconds (Stop-the-world time = 213 milliseconds, Clear Edit Logs time = 125 millis), max Transaction ID 12 2018-08-14 12:27:58,929 INFO [main] o.a.n.c.r.WriteAheadFlowFileRepository Successfully restored 1 FlowFiles 2018-08-14 12:27:58,975 INFO [main] o.a.n.c.repository.FileSystemRepository Found unknown file /root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/content_repository/2/1534231578533-2 (5392 bytes) in File System Repository; archiving file 2018-08-14 12:27:59,010 INFO [main] o.a.n.c.repository.FileSystemRepository Found unknown file /root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/content_repository/1/1534229712680-1 (1511722 bytes) in File System Repository; archiving file 2018-08-14 12:27:59,861 INFO [main] o.apache.nifi.controller.FlowController Starting 0 processors/ports/funnels 2018-08-14 12:27:59,863 INFO [main] o.apache.nifi.controller.FlowController Started 0 Remote Group Ports transmitting 2018-08-14 12:27:59,863 INFO [main] o.a.n.w.c.ApplicationStartupContextListener Flow Controller started successfully. 2018-08-14 12:28:07,676 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@6392107e{/nifi-api,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-web-api-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-web-api-1.5.0.3.1.2.0-7.war} 2018-08-14 12:28:09,299 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=1070ms 2018-08-14 12:28:09,304 INFO [main] o.e.j.C./nifi-content-viewer No Spring WebApplicationInitializer types detected on classpath 2018-08-14 12:28:09,417 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@196ca821{/nifi-content-viewer,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-web-content-viewer-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-web-content-viewer-1.5.0.3.1.2.0-7.war} 2018-08-14 12:28:09,423 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.s.h.ContextHandler@66e478c7{/nifi-docs,null,AVAILABLE} 2018-08-14 12:28:09,533 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=59ms 2018-08-14 12:28:09,628 INFO [main] o.e.jetty.ContextHandler./nifi-docs No Spring WebApplicationInitializer types detected on classpath 2018-08-14 12:28:09,634 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@1a35f99a{/nifi-docs,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-web-docs-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-web-docs-1.5.0.3.1.2.0-7.war} 2018-08-14 12:28:09,730 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=62ms 2018-08-14 12:28:09,799 INFO [main] org.eclipse.jetty.ContextHandler./ No Spring WebApplicationInitializer types detected on classpath 2018-08-14 12:28:09,813 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@72c743f4{/,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-web-error-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-web-error-1.5.0.3.1.2.0-7.war} 2018-08-14 12:28:09,864 INFO [main] o.eclipse.jetty.server.AbstractConnector Started ServerConnector@359eab6a{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} 2018-08-14 12:28:09,864 INFO [main] org.eclipse.jetty.server.Server Started @67825ms 2018-08-14 12:28:09,887 INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs: 2018-08-14 12:28:09,887 INFO [main] org.apache.nifi.web.server.JettyServer http://192.168.100.16:8080/nifi 2018-08-14 12:28:09,887 INFO [main] org.apache.nifi.web.server.JettyServer http://127.0.0.1:8080/nifi 2018-08-14 12:28:09,890 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap 2018-08-14 12:28:09,890 INFO [main] org.apache.nifi.NiFi Controller initialization took 57791450781 nanoseconds (57 seconds). 2018-08-14 12:29:07,091 INFO [NiFi Web Server-16] o.a.n.c.s.StandardProcessScheduler Starting SelectHiveQL[id=a34193cf-57c3-36c9-bb32-46deea363e84] 2018-08-14 12:29:07,116 INFO [NiFi Web Server-16] o.a.n.c.s.StandardProcessScheduler Starting PutHiveStreaming[id=94ac4b79-43b5-39cf-ac60-e665edb3e670] 2018-08-14 12:29:07,129 INFO [Timer-Driven Process Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled SelectHiveQL[id=a34193cf-57c3-36c9-bb32-46deea363e84] to run with 1 threads 2018-08-14 12:29:07,258 INFO [Timer-Driven Process Thread-8] o.a.nifi.dbcp.hive.HiveConnectionPool HiveConnectionPool[id=2ed06e6e-e34f-3b84-39ef-b4637ca2ebab] Simple Authentication 2018-08-14 12:29:07,329 INFO [Timer-Driven Process Thread-8] org.apache.hive.jdbc.Utils Supplied authorities: hdp-1-nn.com:10000 2018-08-14 12:29:07,330 INFO [Timer-Driven Process Thread-8] org.apache.hive.jdbc.Utils Resolved authority: hdp-1-nn.com:10000 2018-08-14 12:29:08,392 INFO [Timer-Driven Process Thread-8] org.apache.hive.jdbc.Utils Supplied authorities: hdp-1-nn.com:10000 2018-08-14 12:29:08,392 INFO [Timer-Driven Process Thread-8] org.apache.hive.jdbc.Utils Resolved authority: hdp-1-nn.com:10000 2018-08-14 12:29:08,471 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@1c76a82c // Another save pending = false 2018-08-14 12:29:10,050 WARN [Timer-Driven Process Thread-7] org.apache.hadoop.util.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-08-14 12:29:11,649 WARN [Timer-Driven Process Thread-7] o.a.h.h.shortcircuit.DomainSocketFactory The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. 2018-08-14 12:29:11,701 INFO [Timer-Driven Process Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled PutHiveStreaming[id=94ac4b79-43b5-39cf-ac60-e665edb3e670] to run with 1 threads 2018-08-14 12:29:46,103 INFO [Timer-Driven Process Thread-8] hive.ql.parse.ParseDriver Parsing command: select HW_2G_Result_Time, HW_2G_Granularity_Period, HW_2G_BSC_Name, HW_2G_Site_Code, HW_2G_Site_Name, HW_2G_Cell_Index, HW_2G_CGI_Name, HW_2G_Reliability, HW_2G_K3000, HW_2G_K3001, HW_2G_K3003, HW_2G_K3004, HW_2G_K3005, HW_2G_K3006, HW_2G_K3010A, HW_2G_K3010B, HW_2G_K3011A, HW_2G_K3011B, HW_2G_K3012A, HW_2G_K3012B, HW_2G_K3013A, HW_2G_K3013B, HW_2G_K3014, HW_2G_K3015, HW_2G_K3016, HW_2G_K3020, HW_2G_K3021, HW_2G_K3022, HW_2G_K3023, HW_2G_K3024, HW_2G_K3034, HW_2G_K3040, HW_2G_K3041, HW_2G_K3043, HW_2G_K3045, HW_2G_K3100, HW_2G_K3101, HW_2G_K3170, HW_2G_K3173F, HW_2G_K3173, HW_2G_RK3255, HW_2G_RM3176, HW_2G_RM3178, HW_2G_RM3179, HW_2G_RM3276, HW_2G_RM3278, HW_2G_RM3279 from (select from_unixtime(unix_timestamp(HW_2G_Result_Time, "yyyy-MM-dd HH:mm")) as HW_2G_Result_Time, HW_2G_Granularity_Period, HW_2G_BSC_Name, HW_2G_Site_Code, HW_2G_Site_Name, HW_2G_Cell_Index, HW_2G_CGI_Name, HW_2G_Reliability, HW_2G_K3000, HW_2G_K3001, HW_2G_K3003, HW_2G_K3004, HW_2G_K3005, HW_2G_K3006, HW_2G_K3010A, HW_2G_K3010B, HW_2G_K3011A, HW_2G_K3011B, HW_2G_K3012A, HW_2G_K3012B, HW_2G_K3013A, HW_2G_K3013B, HW_2G_K3014, HW_2G_K3015, HW_2G_K3016, HW_2G_K3020, HW_2G_K3021, HW_2G_K3022, HW_2G_K3023, HW_2G_K3024, HW_2G_K3034, HW_2G_K3040, HW_2G_K3041, HW_2G_K3043, HW_2G_K3045, HW_2G_K3100, HW_2G_K3101, HW_2G_K3170, HW_2G_K3173F, HW_2G_K3173, HW_2G_RK3255, HW_2G_RM3176, HW_2G_RM3178, HW_2G_RM3179, HW_2G_RM3276, HW_2G_RM3278, HW_2G_RM3279 from ext_ufm_kpi_measurement_per_cell) a inner join timekey b on a.HW_2G_Result_Time = b.timekey and b.tab_name = 'ufm_kpi_measurement_per_cell' 2018-08-14 12:29:47,534 INFO [Timer-Driven Process Thread-8] hive.ql.parse.ParseDriver Parse Completed 2018-08-14 12:29:49,583 INFO [Provenance Maintenance Thread-3] o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers for events starting with ID 23 2018-08-14 12:29:49,807 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.lucene.SimpleIndexManager Index Writer for ./provenance_repository/index-1534229719000 has been returned to Index Manager and is no longer in use. Closing Index Writer 2018-08-14 12:29:49,811 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files (2 records) into single Provenance Log File ./provenance_repository/21.prov in 225 milliseconds 2018-08-14 12:29:49,812 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance Event file containing 1 records. In the past 5 minutes, 2 events have been written to the Provenance Repository, totaling 951 bytes 2018-08-14 12:29:49,961 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:29:58,935 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:29:59,050 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 115 milliseconds (Stop-the-world time = 61 milliseconds, Clear Edit Logs time = 49 millis), max Transaction ID 13 2018-08-14 12:29:59,052 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 116 milliseconds 2018-08-14 12:31:49,969 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:31:59,053 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:31:59,179 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 125 milliseconds (Stop-the-world time = 76 milliseconds, Clear Edit Logs time = 42 millis), max Transaction ID 13 2018-08-14 12:31:59,179 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 126 milliseconds 2018-08-14 12:33:49,976 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:33:59,179 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:33:59,305 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 125 milliseconds (Stop-the-world time = 79 milliseconds, Clear Edit Logs time = 39 millis), max Transaction ID 13 2018-08-14 12:33:59,305 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 125 milliseconds 2018-08-14 12:35:17,872 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@1c76a82c // Another save pending = false 2018-08-14 12:35:24,344 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@1c76a82c // Another save pending = false 2018-08-14 12:35:43,868 INFO [NiFi Web Server-95] o.a.n.controller.StandardFlowFileQueue Canceling ListFlowFile Request with ID 375ca3d2-0165-1000-3892-9a51e6743738 2018-08-14 12:35:49,986 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 9 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:35:59,305 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:35:59,424 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 117 milliseconds (Stop-the-world time = 59 milliseconds, Clear Edit Logs time = 53 millis), max Transaction ID 13 2018-08-14 12:35:59,424 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 118 milliseconds 2018-08-14 12:36:20,874 INFO [NiFi Web Server-91] o.a.n.controller.StandardFlowFileQueue Canceling ListFlowFile Request with ID 375d3483-0165-1000-62dd-c02f2fc07404 2018-08-14 12:36:29,614 INFO [Provenance Maintenance Thread-1] o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers for events starting with ID 24 2018-08-14 12:36:29,651 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.lucene.SimpleIndexManager Index Writer for ./provenance_repository/index-1534229719000 has been returned to Index Manager and is no longer in use. Closing Index Writer 2018-08-14 12:36:29,656 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files (1 records) into single Provenance Log File ./provenance_repository/23.prov in 55 milliseconds 2018-08-14 12:36:29,656 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance Event file containing 1 records. In the past 5 minutes, 1 events have been written to the Provenance Repository, totaling 866 bytes 2018-08-14 12:36:59,645 INFO [Provenance Maintenance Thread-2] o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers for events starting with ID 25 2018-08-14 12:36:59,733 INFO [Provenance Repository Rollover Thread-2] o.a.n.p.lucene.SimpleIndexManager Index Writer for ./provenance_repository/index-1534229719000 has been returned to Index Manager and is no longer in use. Closing Index Writer 2018-08-14 12:36:59,740 INFO [Provenance Repository Rollover Thread-2] o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files (1 records) into single Provenance Log File ./provenance_repository/24.prov in 105 milliseconds 2018-08-14 12:36:59,746 INFO [Provenance Repository Rollover Thread-2] o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance Event file containing 1 records. In the past 5 minutes, 2 events have been written to the Provenance Repository, totaling 1.69 KB 2018-08-14 12:37:49,992 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:37:59,424 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:37:59,562 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 137 milliseconds (Stop-the-world time = 84 milliseconds, Clear Edit Logs time = 44 millis), max Transaction ID 13 2018-08-14 12:37:59,562 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 137 milliseconds 2018-08-14 12:39:50,000 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 7 milliseconds (Stop-the-world time = 3 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:39:59,562 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:39:59,706 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 142 milliseconds (Stop-the-world time = 84 milliseconds, Clear Edit Logs time = 53 millis), max Transaction ID 13 2018-08-14 12:39:59,706 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 143 milliseconds 2018-08-14 12:41:50,007 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:41:59,706 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:41:59,817 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 110 milliseconds (Stop-the-world time = 52 milliseconds, Clear Edit Logs time = 54 millis), max Transaction ID 13 2018-08-14 12:41:59,817 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 110 milliseconds 2018-08-14 12:43:50,013 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 5 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:43:59,817 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:43:59,949 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 131 milliseconds (Stop-the-world time = 88 milliseconds, Clear Edit Logs time = 38 millis), max Transaction ID 13 2018-08-14 12:43:59,949 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 131 milliseconds 2018-08-14 12:45:50,020 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1

Looking forward.