Support Questions
Find answers, ask questions, and share your expertise

NiFi: PutHiveStreaming is not loading data into Hive Managed table without any error.

NiFi: PutHiveStreaming is not loading data into Hive Managed table without any error.

Nifi not loading any thing in target Hive Managed Table (no partition, yes buckets) from Hive External Tables.

Nifi Processors:

  1. SelectHiveQL: Running successfully and sends out a Queue.
  2. PutHiveStreaming: Not able to pull Queue from SelectHiveQL processor. There is no error, it just keep running.

Already started NiFi.

Please advice.

Below is latest log:

=====================================================================

2018-08-14 12:27:23,215 INFO [main] org.eclipse.jetty.server.Server jetty-9.4.3.v20170317 2018-08-14 12:27:28,889 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=4651ms 2018-08-14 12:27:29,299 INFO [main] org.eclipse.jetty.server.session DefaultSessionIdManager workerName=node0 2018-08-14 12:27:29,299 INFO [main] org.eclipse.jetty.server.session No SessionScavenger set, using defaults 2018-08-14 12:27:29,301 INFO [main] org.eclipse.jetty.server.session Scavenging every 660000ms 2018-08-14 12:27:29,415 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@7d626f2{/nifi-jolt-transform-json-ui-1.5.0.3.1.2.0-7,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-jolt-transform-json-ui-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/extensions/nifi-standard-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-jolt-transform-json-ui-1.5.0.3.1.2.0-7.war} 2018-08-14 12:27:33,702 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=3744ms 2018-08-14 12:27:33,855 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@2c08c787{/nifi-update-attribute-ui-1.5.0.3.1.2.0-7,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-update-attribute-ui-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/extensions/nifi-update-attribute-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-update-attribute-ui-1.5.0.3.1.2.0-7.war} 2018-08-14 12:27:35,522 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=1264ms 2018-08-14 12:27:35,632 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@302da330{/nifi-standard-content-viewer-1.5.0.3.1.2.0-7,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-standard-content-viewer-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/extensions/nifi-standard-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-standard-content-viewer-1.5.0.3.1.2.0-7.war} 2018-08-14 12:27:35,727 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=63ms 2018-08-14 12:27:35,834 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@7c9372ed{/nifi-image-viewer-1.5.0.3.1.2.0-7,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-image-viewer-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/extensions/nifi-media-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-image-viewer-1.5.0.3.1.2.0-7.war} 2018-08-14 12:27:36,308 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=165ms 2018-08-14 12:27:36,333 INFO [main] org.eclipse.jetty.ContextHandler./nifi No Spring WebApplicationInitializer types detected on classpath 2018-08-14 12:27:36,467 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@4387d2f4{/nifi,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-web-ui-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-web-ui-1.5.0.3.1.2.0-7.war} 2018-08-14 12:27:36,816 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=175ms 2018-08-14 12:27:36,929 INFO [main] o.eclipse.jetty.ContextHandler./nifi-api No Spring WebApplicationInitializer types detected on classpath 2018-08-14 12:27:36,956 INFO [main] o.eclipse.jetty.ContextHandler./nifi-api Initializing Spring root WebApplicationContext 2018-08-14 12:27:44,383 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be '/root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/./conf/nifi.properties' 2018-08-14 12:27:44,391 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be '/root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/./conf/nifi.properties' 2018-08-14 12:27:44,392 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 145 properties from /root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/./conf/nifi.properties 2018-08-14 12:27:48,788 INFO [main] o.a.n.r.v.FileBasedVariableRegistry Loaded 116 properties from system properties and environment variables 2018-08-14 12:27:48,788 INFO [main] o.a.n.r.v.FileBasedVariableRegistry Loaded a total of 116 properties. Including precedence overrides effective accessible registry key size is 116 2018-08-14 12:27:48,980 INFO [main] o.a.n.c.r.WriteAheadFlowFileRepository Initialized FlowFile Repository using 256 partitions 2018-08-14 12:27:49,516 INFO [main] o.a.n.p.lucene.SimpleIndexManager Index Writer for ./provenance_repository/index-1534229719000 has been returned to Index Manager and is no longer in use. Closing Index Writer 2018-08-14 12:27:49,529 INFO [main] o.a.n.p.PersistentProvenanceRepository Recovered 21 records 2018-08-14 12:27:49,561 INFO [main] o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers for events starting with ID 21 2018-08-14 12:27:49,577 INFO [main] o.a.n.c.repository.FileSystemRepository Maximum Threshold for Container default set to 53599786106 bytes; if volume exceeds this size, archived data will be deleted until it no longer exceeds this size 2018-08-14 12:27:49,578 INFO [main] o.a.n.c.repository.FileSystemRepository Initializing FileSystemRepository with 'Always Sync' set to false 2018-08-14 12:27:49,931 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 finished recovering records. Performing Checkpoint to ensure proper state of Partitions before updates 2018-08-14 12:27:49,932 INFO [main] org.wali.MinimalLockingWriteAheadLog Successfully recovered 0 records in 14 milliseconds 2018-08-14 12:27:49,953 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 18 milliseconds (Stop-the-world time = 5 milliseconds, Clear Edit Logs time = 8 millis), max Transaction ID -1 2018-08-14 12:27:50,040 INFO [main] o.apache.nifi.controller.FlowController Not enabling RAW Socket Site-to-Site functionality because nifi.remote.input.socket.port is not set 2018-08-14 12:27:50,143 INFO [main] o.a.n.w.c.ApplicationStartupContextListener Starting Flow Controller... 2018-08-14 12:27:51,860 INFO [main] o.a.n.c.s.StandardControllerServiceProvider Created Controller Service of type org.apache.nifi.dbcp.hive.HiveConnectionPool with identifier 2ed06e6e-e34f-3b84-39ef-b4637ca2ebab 2018-08-14 12:27:51,881 INFO [main] o.a.nifi.groups.StandardProcessGroup HiveConnectionPool[id=2ed06e6e-e34f-3b84-39ef-b4637ca2ebab] added to StandardProcessGroup[identifier=3731b8ce-0165-1000-e94c-aaff03b543dd] 2018-08-14 12:27:53,580 INFO [main] o.a.n.c.s.StandardProcessScheduler Enabling HiveConnectionPool[id=2ed06e6e-e34f-3b84-39ef-b4637ca2ebab] 2018-08-14 12:27:57,256 INFO [main] o.a.nifi.groups.StandardProcessGroup Template[id=7a6ac69b-0ef8-4033-ad30-5ecf62507636] added to StandardProcessGroup[identifier=3731b8ce-0165-1000-e94c-aaff03b543dd] 2018-08-14 12:27:58,077 INFO [main] o.apache.nifi.controller.FlowController Successfully synchronized controller with proposed flow 2018-08-14 12:27:58,578 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 finished recovering records. Performing Checkpoint to ensure proper state of Partitions before updates 2018-08-14 12:27:58,578 INFO [main] org.wali.MinimalLockingWriteAheadLog Successfully recovered 1 records in 325 milliseconds 2018-08-14 12:27:58,927 INFO [main] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 1 Records and 0 Swap Files in 348 milliseconds (Stop-the-world time = 213 milliseconds, Clear Edit Logs time = 125 millis), max Transaction ID 12 2018-08-14 12:27:58,929 INFO [main] o.a.n.c.r.WriteAheadFlowFileRepository Successfully restored 1 FlowFiles 2018-08-14 12:27:58,975 INFO [main] o.a.n.c.repository.FileSystemRepository Found unknown file /root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/content_repository/2/1534231578533-2 (5392 bytes) in File System Repository; archiving file 2018-08-14 12:27:59,010 INFO [main] o.a.n.c.repository.FileSystemRepository Found unknown file /root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/content_repository/1/1534229712680-1 (1511722 bytes) in File System Repository; archiving file 2018-08-14 12:27:59,861 INFO [main] o.apache.nifi.controller.FlowController Starting 0 processors/ports/funnels 2018-08-14 12:27:59,863 INFO [main] o.apache.nifi.controller.FlowController Started 0 Remote Group Ports transmitting 2018-08-14 12:27:59,863 INFO [main] o.a.n.w.c.ApplicationStartupContextListener Flow Controller started successfully. 2018-08-14 12:28:07,676 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@6392107e{/nifi-api,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-web-api-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-web-api-1.5.0.3.1.2.0-7.war} 2018-08-14 12:28:09,299 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=1070ms 2018-08-14 12:28:09,304 INFO [main] o.e.j.C./nifi-content-viewer No Spring WebApplicationInitializer types detected on classpath 2018-08-14 12:28:09,417 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@196ca821{/nifi-content-viewer,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-web-content-viewer-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-web-content-viewer-1.5.0.3.1.2.0-7.war} 2018-08-14 12:28:09,423 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.s.h.ContextHandler@66e478c7{/nifi-docs,null,AVAILABLE} 2018-08-14 12:28:09,533 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=59ms 2018-08-14 12:28:09,628 INFO [main] o.e.jetty.ContextHandler./nifi-docs No Spring WebApplicationInitializer types detected on classpath 2018-08-14 12:28:09,634 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@1a35f99a{/nifi-docs,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-web-docs-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-web-docs-1.5.0.3.1.2.0-7.war} 2018-08-14 12:28:09,730 INFO [main] o.e.j.a.AnnotationConfiguration Scanning elapsed time=62ms 2018-08-14 12:28:09,799 INFO [main] org.eclipse.jetty.ContextHandler./ No Spring WebApplicationInitializer types detected on classpath 2018-08-14 12:28:09,813 INFO [main] o.e.jetty.server.handler.ContextHandler Started o.e.j.w.WebAppContext@72c743f4{/,file:///root/Desktop/nifi-1.5.0.3.1.2.0-7-bin/nifi-1.5.0.3.1.2.0-7/work/jetty/nifi-web-error-1.5.0.3.1.2.0-7.war/webapp/,AVAILABLE}{./work/nar/framework/nifi-framework-nar-1.5.0.3.1.2.0-7.nar-unpacked/META-INF/bundled-dependencies/nifi-web-error-1.5.0.3.1.2.0-7.war} 2018-08-14 12:28:09,864 INFO [main] o.eclipse.jetty.server.AbstractConnector Started ServerConnector@359eab6a{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} 2018-08-14 12:28:09,864 INFO [main] org.eclipse.jetty.server.Server Started @67825ms 2018-08-14 12:28:09,887 INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs: 2018-08-14 12:28:09,887 INFO [main] org.apache.nifi.web.server.JettyServer http://192.168.100.16:8080/nifi 2018-08-14 12:28:09,887 INFO [main] org.apache.nifi.web.server.JettyServer http://127.0.0.1:8080/nifi 2018-08-14 12:28:09,890 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap 2018-08-14 12:28:09,890 INFO [main] org.apache.nifi.NiFi Controller initialization took 57791450781 nanoseconds (57 seconds). 2018-08-14 12:29:07,091 INFO [NiFi Web Server-16] o.a.n.c.s.StandardProcessScheduler Starting SelectHiveQL[id=a34193cf-57c3-36c9-bb32-46deea363e84] 2018-08-14 12:29:07,116 INFO [NiFi Web Server-16] o.a.n.c.s.StandardProcessScheduler Starting PutHiveStreaming[id=94ac4b79-43b5-39cf-ac60-e665edb3e670] 2018-08-14 12:29:07,129 INFO [Timer-Driven Process Thread-6] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled SelectHiveQL[id=a34193cf-57c3-36c9-bb32-46deea363e84] to run with 1 threads 2018-08-14 12:29:07,258 INFO [Timer-Driven Process Thread-8] o.a.nifi.dbcp.hive.HiveConnectionPool HiveConnectionPool[id=2ed06e6e-e34f-3b84-39ef-b4637ca2ebab] Simple Authentication 2018-08-14 12:29:07,329 INFO [Timer-Driven Process Thread-8] org.apache.hive.jdbc.Utils Supplied authorities: hdp-1-nn.com:10000 2018-08-14 12:29:07,330 INFO [Timer-Driven Process Thread-8] org.apache.hive.jdbc.Utils Resolved authority: hdp-1-nn.com:10000 2018-08-14 12:29:08,392 INFO [Timer-Driven Process Thread-8] org.apache.hive.jdbc.Utils Supplied authorities: hdp-1-nn.com:10000 2018-08-14 12:29:08,392 INFO [Timer-Driven Process Thread-8] org.apache.hive.jdbc.Utils Resolved authority: hdp-1-nn.com:10000 2018-08-14 12:29:08,471 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@1c76a82c // Another save pending = false 2018-08-14 12:29:10,050 WARN [Timer-Driven Process Thread-7] org.apache.hadoop.util.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-08-14 12:29:11,649 WARN [Timer-Driven Process Thread-7] o.a.h.h.shortcircuit.DomainSocketFactory The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. 2018-08-14 12:29:11,701 INFO [Timer-Driven Process Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled PutHiveStreaming[id=94ac4b79-43b5-39cf-ac60-e665edb3e670] to run with 1 threads 2018-08-14 12:29:46,103 INFO [Timer-Driven Process Thread-8] hive.ql.parse.ParseDriver Parsing command: select HW_2G_Result_Time, HW_2G_Granularity_Period, HW_2G_BSC_Name, HW_2G_Site_Code, HW_2G_Site_Name, HW_2G_Cell_Index, HW_2G_CGI_Name, HW_2G_Reliability, HW_2G_K3000, HW_2G_K3001, HW_2G_K3003, HW_2G_K3004, HW_2G_K3005, HW_2G_K3006, HW_2G_K3010A, HW_2G_K3010B, HW_2G_K3011A, HW_2G_K3011B, HW_2G_K3012A, HW_2G_K3012B, HW_2G_K3013A, HW_2G_K3013B, HW_2G_K3014, HW_2G_K3015, HW_2G_K3016, HW_2G_K3020, HW_2G_K3021, HW_2G_K3022, HW_2G_K3023, HW_2G_K3024, HW_2G_K3034, HW_2G_K3040, HW_2G_K3041, HW_2G_K3043, HW_2G_K3045, HW_2G_K3100, HW_2G_K3101, HW_2G_K3170, HW_2G_K3173F, HW_2G_K3173, HW_2G_RK3255, HW_2G_RM3176, HW_2G_RM3178, HW_2G_RM3179, HW_2G_RM3276, HW_2G_RM3278, HW_2G_RM3279 from (select from_unixtime(unix_timestamp(HW_2G_Result_Time, "yyyy-MM-dd HH:mm")) as HW_2G_Result_Time, HW_2G_Granularity_Period, HW_2G_BSC_Name, HW_2G_Site_Code, HW_2G_Site_Name, HW_2G_Cell_Index, HW_2G_CGI_Name, HW_2G_Reliability, HW_2G_K3000, HW_2G_K3001, HW_2G_K3003, HW_2G_K3004, HW_2G_K3005, HW_2G_K3006, HW_2G_K3010A, HW_2G_K3010B, HW_2G_K3011A, HW_2G_K3011B, HW_2G_K3012A, HW_2G_K3012B, HW_2G_K3013A, HW_2G_K3013B, HW_2G_K3014, HW_2G_K3015, HW_2G_K3016, HW_2G_K3020, HW_2G_K3021, HW_2G_K3022, HW_2G_K3023, HW_2G_K3024, HW_2G_K3034, HW_2G_K3040, HW_2G_K3041, HW_2G_K3043, HW_2G_K3045, HW_2G_K3100, HW_2G_K3101, HW_2G_K3170, HW_2G_K3173F, HW_2G_K3173, HW_2G_RK3255, HW_2G_RM3176, HW_2G_RM3178, HW_2G_RM3179, HW_2G_RM3276, HW_2G_RM3278, HW_2G_RM3279 from ext_ufm_kpi_measurement_per_cell) a inner join timekey b on a.HW_2G_Result_Time = b.timekey and b.tab_name = 'ufm_kpi_measurement_per_cell' 2018-08-14 12:29:47,534 INFO [Timer-Driven Process Thread-8] hive.ql.parse.ParseDriver Parse Completed 2018-08-14 12:29:49,583 INFO [Provenance Maintenance Thread-3] o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers for events starting with ID 23 2018-08-14 12:29:49,807 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.lucene.SimpleIndexManager Index Writer for ./provenance_repository/index-1534229719000 has been returned to Index Manager and is no longer in use. Closing Index Writer 2018-08-14 12:29:49,811 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files (2 records) into single Provenance Log File ./provenance_repository/21.prov in 225 milliseconds 2018-08-14 12:29:49,812 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance Event file containing 1 records. In the past 5 minutes, 2 events have been written to the Provenance Repository, totaling 951 bytes 2018-08-14 12:29:49,961 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:29:58,935 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:29:59,050 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 115 milliseconds (Stop-the-world time = 61 milliseconds, Clear Edit Logs time = 49 millis), max Transaction ID 13 2018-08-14 12:29:59,052 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 116 milliseconds 2018-08-14 12:31:49,969 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:31:59,053 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:31:59,179 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 125 milliseconds (Stop-the-world time = 76 milliseconds, Clear Edit Logs time = 42 millis), max Transaction ID 13 2018-08-14 12:31:59,179 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 126 milliseconds 2018-08-14 12:33:49,976 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:33:59,179 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:33:59,305 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 125 milliseconds (Stop-the-world time = 79 milliseconds, Clear Edit Logs time = 39 millis), max Transaction ID 13 2018-08-14 12:33:59,305 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 125 milliseconds 2018-08-14 12:35:17,872 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@1c76a82c // Another save pending = false 2018-08-14 12:35:24,344 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@1c76a82c // Another save pending = false 2018-08-14 12:35:43,868 INFO [NiFi Web Server-95] o.a.n.controller.StandardFlowFileQueue Canceling ListFlowFile Request with ID 375ca3d2-0165-1000-3892-9a51e6743738 2018-08-14 12:35:49,986 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 9 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:35:59,305 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:35:59,424 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 117 milliseconds (Stop-the-world time = 59 milliseconds, Clear Edit Logs time = 53 millis), max Transaction ID 13 2018-08-14 12:35:59,424 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 118 milliseconds 2018-08-14 12:36:20,874 INFO [NiFi Web Server-91] o.a.n.controller.StandardFlowFileQueue Canceling ListFlowFile Request with ID 375d3483-0165-1000-62dd-c02f2fc07404 2018-08-14 12:36:29,614 INFO [Provenance Maintenance Thread-1] o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers for events starting with ID 24 2018-08-14 12:36:29,651 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.lucene.SimpleIndexManager Index Writer for ./provenance_repository/index-1534229719000 has been returned to Index Manager and is no longer in use. Closing Index Writer 2018-08-14 12:36:29,656 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files (1 records) into single Provenance Log File ./provenance_repository/23.prov in 55 milliseconds 2018-08-14 12:36:29,656 INFO [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance Event file containing 1 records. In the past 5 minutes, 1 events have been written to the Provenance Repository, totaling 866 bytes 2018-08-14 12:36:59,645 INFO [Provenance Maintenance Thread-2] o.a.n.p.PersistentProvenanceRepository Created new Provenance Event Writers for events starting with ID 25 2018-08-14 12:36:59,733 INFO [Provenance Repository Rollover Thread-2] o.a.n.p.lucene.SimpleIndexManager Index Writer for ./provenance_repository/index-1534229719000 has been returned to Index Manager and is no longer in use. Closing Index Writer 2018-08-14 12:36:59,740 INFO [Provenance Repository Rollover Thread-2] o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal files (1 records) into single Provenance Log File ./provenance_repository/24.prov in 105 milliseconds 2018-08-14 12:36:59,746 INFO [Provenance Repository Rollover Thread-2] o.a.n.p.PersistentProvenanceRepository Successfully Rolled over Provenance Event file containing 1 records. In the past 5 minutes, 2 events have been written to the Provenance Repository, totaling 1.69 KB 2018-08-14 12:37:49,992 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:37:59,424 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:37:59,562 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 137 milliseconds (Stop-the-world time = 84 milliseconds, Clear Edit Logs time = 44 millis), max Transaction ID 13 2018-08-14 12:37:59,562 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 137 milliseconds 2018-08-14 12:39:50,000 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 7 milliseconds (Stop-the-world time = 3 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:39:59,562 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:39:59,706 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 142 milliseconds (Stop-the-world time = 84 milliseconds, Clear Edit Logs time = 53 millis), max Transaction ID 13 2018-08-14 12:39:59,706 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 143 milliseconds 2018-08-14 12:41:50,007 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:41:59,706 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:41:59,817 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 110 milliseconds (Stop-the-world time = 52 milliseconds, Clear Edit Logs time = 54 millis), max Transaction ID 13 2018-08-14 12:41:59,817 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 110 milliseconds 2018-08-14 12:43:50,013 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 5 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1 2018-08-14 12:43:59,817 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository 2018-08-14 12:43:59,949 INFO [pool-10-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@87f8146 checkpointed with 2 Records and 0 Swap Files in 131 milliseconds (Stop-the-world time = 88 milliseconds, Clear Edit Logs time = 38 millis), max Transaction ID 13 2018-08-14 12:43:59,949 INFO [pool-10-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 131 milliseconds 2018-08-14 12:45:50,020 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@7097ead6 checkpointed with 0 Records and 0 Swap Files in 6 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID -1

Looking forward.

6 REPLIES 6

Re: NiFi: PutHiveStreaming is not loading data into Hive Managed table without any error.

Realized PutHiveStreaming processing need to be scheduled=0s.

Now it is pulling queue and showing successful but nothing been loaded in target Hive Managed table.

Strange, how it is possible that its showing successful but nothing is getting loaded.

Please advice.

Re: NiFi: PutHiveStreaming is not loading data into Hive Managed table without any error.

Super Guru

The log above doesn't show PutHiveStreaming actually running, just that it was scheduled. Do you see a number in the upper right hand corner of the PutHiveStreaming processor? That number is the number of processor threads currently doing work. If your Metastore URI is incorrect or some other configuration property is incorrect, it could be trying "forever" to connect to the Hive Metastore. Try checking the Hive Metastore logs while PutHiveStreaming is running, to see if NiFi is connecting to it correctly,

Re: NiFi: PutHiveStreaming is not loading data into Hive Managed table without any error.

Thanks Matt for jumping in. Noticed, you helping on the same issue on other forums, very hopeful here as well :-) as we are stuck on this issue since one week and unable to move ahead.

Environment

  • HDP 3.0
  • NiFi 1.5: Downloaded from Hortonworks site
  • Both are on same node, using same root user, so can't be permission issue

Addition to issue: (As issue#2 is of same nature, need to resolve both which will help others user in one-go.)

  1. When we are not using Partition, then everything runs fine but nothing gets loaded. Shared the log above.
  2. When we are using Partition, following error coming

==================================

2018-08-15 20:18:17,477 ERROR [Timer-Driven Process Thread-4] o.a.n.processors.hive.PutHiveStreaming PutHiveStreaming[id=5b4fddea-bea1-3665-fcaf-d824a2cbb3c4] Error connecting to Hive endpoint: table ufm_intra_cell_handover_measurement_per_cell at thrift://hdp-1-nn.com:9083 2018-08-15 20:18:17,477 ERROR [Timer-Driven Process Thread-4] o.a.n.processors.hive.PutHiveStreaming PutHiveStreaming[id=5b4fddea-bea1-3665-fcaf-d824a2cbb3c4] Hive Streaming connect/write error, flow file will be penalized and routed to retry. org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://hdp-1-nn.com:9083', database='ufm', table='ufm_intra_cell_handover_measurement_per_cell', partitionVals=[ISB009H__I10_ISB_9] }: org.apache.nifi.processors.hive.PutHiveStreaming$ShouldRetryException: Hive Streaming connect/write error, flow file will be penalized and routed to retry. org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://hdp-1-nn.com:9083', database='ufm', table='ufm_intra_cell_handover_measurement_per_cell', partitionVals=[ISB009H__I10_ISB_9] } org.apache.nifi.processors.hive.PutHiveStreaming$ShouldRetryException: Hive Streaming connect/write error, flow file will be penalized and routed to retry. org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://hdp-1-nn.com:9083', database='ufm', table='ufm_intra_cell_handover_measurement_per_cell', partitionVals=[ISB009H__I10_ISB_9] } at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onHiveRecordsError$1(PutHiveStreaming.java:556) at org.apache.nifi.processor.util.pattern.ExceptionHandler$OnError.lambda$andThen$0(ExceptionHandler.java:54) at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onHiveRecordError$2(PutHiveStreaming.java:574) at org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:148) at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onTrigger$12(PutHiveStreaming.java:752) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2175) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2145) at org.apache.nifi.processors.hive.PutHiveStreaming.onTrigger(PutHiveStreaming.java:706) at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onTrigger$4(PutHiveStreaming.java:584) at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114) at org.apache.nifi.processor.util.pattern.RollbackOnFailure.onTrigger(RollbackOnFailure.java:184) at org.apache.nifi.processors.hive.PutHiveStreaming.onTrigger(PutHiveStreaming.java:584) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1124) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147) at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.nifi.util.hive.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://hdp-1-nn.com:9083', database='ufm', table='ufm_intra_cell_handover_measurement_per_cell', partitionVals=[ISB009H__I10_ISB_9] } at org.apache.nifi.util.hive.HiveWriter.<init>(HiveWriter.java:79) at org.apache.nifi.util.hive.HiveUtils.makeHiveWriter(HiveUtils.java:46) at org.apache.nifi.processors.hive.PutHiveStreaming.makeHiveWriter(PutHiveStreaming.java:1048) at org.apache.nifi.processors.hive.PutHiveStreaming.getOrCreateWriter(PutHiveStreaming.java:959) at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$null$8(PutHiveStreaming.java:755) at org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:127) ... 19 common frames omitted Caused by: org.apache.hive.hcatalog.streaming.StreamingException: partition values=[ISB009H__I10_ISB_9]. Unable to get path for end point: [ISB009H__I10_ISB_9] at org.apache.hive.hcatalog.streaming.AbstractRecordWriter.getPathForEndPoint(AbstractRecordWriter.java:280) at org.apache.hive.hcatalog.streaming.AbstractRecordWriter.<init>(AbstractRecordWriter.java:98) at org.apache.hive.hcatalog.streaming.StrictJsonWriter.<init>(StrictJsonWriter.java:82) at org.apache.hive.hcatalog.streaming.StrictJsonWriter.<init>(StrictJsonWriter.java:60) at org.apache.nifi.util.hive.HiveWriter.getRecordWriter(HiveWriter.java:85) at org.apache.nifi.util.hive.HiveWriter.<init>(HiveWriter.java:72) ... 24 common frames omitted Caused by: org.apache.hadoop.hive.metastore.api.NoSuchObjectException: partition values=[ISB009H__I10_ISB_9] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partition_result$get_partition_resultStandardScheme.read(ThriftHiveMetastore.java) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partition_result$get_partition_resultStandardScheme.read(ThriftHiveMetastore.java) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_partition_result.read(ThriftHiveMetastore.java) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_partition(ThriftHiveMetastore.java:1917) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_partition(ThriftHiveMetastore.java:1902) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getPartition(HiveMetaStoreClient.java:1271) at sun.reflect.GeneratedMethodAccessor417.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:174) at com.sun.proxy.$Proxy138.getPartition(Unknown Source) at org.apache.hive.hcatalog.streaming.AbstractRecordWriter.getPathForEndPoint(AbstractRecordWriter.java:275) ... 29 common frames omitted 2018-08-15 20:18:18,685 WARN [put-hive-streaming-0] hive.metastore Unexpected increment of user count beyond one: 15766 HCatClient: thread: 90 users=15766 expired=false closed=false 2018-08-15 20:18:18,689 WARN [put-hive-streaming-0] hive.metastore Unexpected increment of user count beyond one: 15767 HCatClient: thread: 90 users=15767 expired=false closed=false 2018-08-15 20:18:18,711 WARN [Timer-Driven Process Thread-4] hive.metastore Unexpected increment of user count beyond one: 218 HCatClient: thread: 56 users=218 expired=false closed=false

============================================================

Looking forward.

Re: NiFi: PutHiveStreaming is not loading data into Hive Managed table without any error.

@Matt Burgess

Hive Metastore Log:

hivemetastorelog-tail500rows.txt

Snapshots:

85636-processors.jpeg

85637-selecthiveql.jpeg

85638-puthivestreaming.jpeg

85639-puthivestreaming-error.jpg

Re: NiFi: PutHiveStreaming is not loading data into Hive Managed table without any error.

Re: NiFi: PutHiveStreaming is not loading data into Hive Managed table without any error.

I'm facing the same issue with a similar environment (HDP 3.0 Nifi 1.5). Was it resolved?