Reply
Highlighted
Explorer
Posts: 33
Registered: ‎03-08-2016

Display the characteristics of csv file using Flume (spool directory as source)

I am trying to display the characteristics of csv files (file name, MD5). I don't know how to do it, and is it possible. Any help please

Cloudera Employee
Posts: 277
Registered: ‎01-09-2014

Re: Display the characteristics of csv file using Flume (spool directory as source)

For filename, you can use basenameHeader or fileHeader [1]. There isn't currently a header value that is populated for MD5 however, so you'd have to modify the spooldir source if you needed to pull the md5 value from the file.

-PD

[1] http://flume.apache.org/FlumeUserGuide.html#spooling-directory-source