question Flume - sink values if it is hdfs: http://hortonworks.com/blog/configure-elastic-search-hadoop-hdp-2-0/ "flume example" in Support Questions

Flume - sink values if it is hdfs: http://hortonworks.com/blog/configure-elastic-search-hadoop-hdp-2-0/ "flume example"

Eukrev — Sun, 10 Apr 2016 22:24:42 GMT

Hi,

I got this link from the before post:

http://hortonworks.com/blog/configure-elastic-search-hadoop-hdp-2-0/ "flume example"

The diagram show the sink can be hdfs or elastic search. I would like to try with hdfs in sandbox.

What should be the value of sink for hdfs. I tried sink.type=logger, the configuration file is throwing an error.

Since I am learning flume by trying out the examples mentioned in sandbox, it would be good if you could help me.

Thank you, Revathy.

Re: Flume - sink values if it is hdfs: http://hortonworks.com/blog/configure-elastic-search-hadoop-hdp-2-0/ "flume example"

aervits — Mon, 11 Apr 2016 00:24:15 GMT

Here's all info on Flume sinks and sample hdfs sink configuration

https://flume.apache.org/FlumeUserGuide.html#hdfs-sink

a1.channels = c1
a1.sinks = k1
a1.sinks.k1.type = hdfs
a1.sinks.k1.channel = c1
a1.sinks.k1.hdfs.path = /flume/events/%y-%m-%d/%H%M/%S
a1.sinks.k1.hdfs.filePrefix = events-
a1.sinks.k1.hdfs.round = true
a1.sinks.k1.hdfs.roundValue = 10
a1.sinks.k1.hdfs.roundUnit = minute