Created on 07-05-2019 06:37 PM - edited 08-17-2019 05:00 PM
Hi All,
I am using HDF 3.4.0, I have designed NIFI flow to read events using NETCAT and dump into HDFS .
1. I'm using FlumeSource and FlumeSink processors in NIFI. When I execute the flow, events are being read by source but the sink is not able to write event into HDFS. Below is error in nifi-app.log
Here is source and sink configurations
Source
Sink
2. Also nifi-app.log file shows multiple start and stop for NETCAT source can someone suggest on cause ? Ideally it should be once.
2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source stopping 2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source starting 2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/host:44444] 2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source stopping 2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source starting 2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/host:44444] 2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source stopping 2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source starting 2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/host:44444]
Created 07-06-2019 05:32 PM
The above question was originally posted in the Community Help track. On Sat Jul 6 17:30 UTC 2019, a member of the HCC moderation staff moved it to the Data Ingestion & Streaming track. The Community Help Track is intended for questions about using the HCC site itself, not technical questions about HDF, HFDS, Nifi or using the FlumeSource or FlumeSink processors.
Created 07-08-2019 06:46 AM
Can someone please advice on mentioned issue, I just can't see data in HDFS
FileSystem's output stream doesn't support getNumCurrentReplicas; --HDFS-826 not available; fsOut=org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer; err=java.lang.NoSuchMethodException: org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.getNumCurrentReplicas()