Reply
New Contributor
Posts: 1
Registered: ‎01-16-2014

Flume for Windows?

Does Cloudera no longer support Flume for Windows? The only information I can find about this is from CDH3. (On the Cloudera Archive site) I can't find a single word mentioning it in the 1.x releases or CDH4 documents. I just need to get files from a windows box into HDFS. From my reading, flume seems to be the perfect candidate. 

 

Thanks

Explorer
Posts: 23
Registered: ‎02-11-2014

Re: Flume for Windows?

[ Edited ]

Hello,

 

You can download flume from this link. https://github.com/apache/flume/releases and perform the steps mentioned in README file in downloaded zip file.

 

And after that you can run/check flume with below information

 

You've to edit the log4j.properties file too, simple change the logger facility from 
flume.root.logger=INFO,LOGFILE
into
flume.root.logger=DEBUG,CONSOLE
 
Start Flume:
WINDOW_SHELL>"<<java_path>>"/java.exe -Xmx20m -Dlog4j.configuration=file:///%CD%\conf\log4j.properties -cp "<<FLUME-PATH>>\lib\*" org.apache.flume.node.Application -f <<FLUME-PATH>>\conf\test1.conf -n syslog-agent

Check your flume log file, you should able to see flume service start logs. Once flume is up and running try to post some text on the port which flume is listening.
 
For Example:
echo "<13>any syslog/text message"> /tmp/check-log
nc -v <<WINDOWS-SERVER-IP>> <<port<< < /tmp/check-log
 
On the Windows Box you'll see something like:
2012-07-02 11:36:38,114 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process (LoggerSink.java:70)] Event: { headers:{timestamp=1340187132000, Severity=5, host=host, Facility=8} body: 77 AF 5A 2C 12 34 35 8F A4 10 51 AF QA 80 83 6B any syslog/test mes }
 
Regards,
Chirag Patadia.
New Contributor
Posts: 1
Registered: ‎09-25-2018

Re: Flume for Windows?

which version of flume is compatible with hadoop2.7 in windows ? can you explain complete installation procedure ?

Announcements