NIFI - Flume HDFS sink not working

[#1]

Hi All,

I am using HDF 3.4.0, I have designed NIFI flow to read events using NETCAT and dump into HDFS .

1. I'm using FlumeSource and FlumeSink processors in NIFI. When I execute the flow, events are being read by source but the sink is not able to write event into HDFS. Below is error in nifi-app.log

Here is source and sink configurations

Source

Sink

2. Also nifi-app.log file shows multiple start and stop for NETCAT source can someone suggest on cause ? Ideally it should be once.

2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source stopping
2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source starting
2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/host:44444]
2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source stopping
2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source starting
2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/host:44444]
2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source stopping
2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Source starting
2019-07-05 17:11:11,341 INFO [Timer-Driven Process Thread-4] org.apache.flume.source.NetcatSource Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/host:44444]

[#2]

The above question was originally posted in the Community Help track. On Sat Jul 6 17:30 UTC 2019, a member of the HCC moderation staff moved it to the Data Ingestion & Streaming track. The Community Help Track is intended for questions about using the HCC site itself, not technical questions about HDF, HFDS, Nifi or using the FlumeSource or FlumeSink processors.

[#3]

Can someone please advice on mentioned issue, I just can't see data in HDFS

FileSystem's output stream doesn't support getNumCurrentReplicas; --HDFS-826 not available; fsOut=org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer; err=java.lang.NoSuchMethodException: org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.getNumCurrentReplicas()