Flume Tutorials

[#1]

Are there any good walkthrough tutorials for Flume? I've seen the two listed here. However, after skimming through the second one "Analyzing Social Media and Customer Sentiment," I fail to see any use or reference of Flume within it. I would specifically like something that walks through performance of the two Flume objectives documented in the HDP Certified Developer Exam Objectives sheet:

1. Given a Flume configuration file, start a Flume agent
2. Given a configured sink and source, configure a Flume memory channel with a specified capacity https

The 1st tutorial from the link above starts a Flume agent via Ambari, but I assume the Exam will require this to be done via the Terminal.

[#2]

@Daniel Hendrix

This is good tutorial http://hortonworks.com/blog/configure-elastic-search-hadoop-hdp-2-0/ "flume example"

[#3]

@Neeraj Sabharwal Thanks! I hadn't ran across this one yet.

[#4]

Hi Daniel,

Were you able to run this flume example? Because I am trying. What would be the values if the sink is hdfs and not elasticsearch? any idea

Thank you.

[#5]

@rich

[#6]

@Artem Ervits Thanks, this is very helpful.

[#7]

Hi @Daniel Hendrix

You are correct - you should be able to start a Flume agent from the command line. The docs show how to do this:

https://flume.apache.org/FlumeUserGuide.html#starting-an-agent

You also need to know how to configure a memory channel, which is also demonstrated in the docs:

https://flume.apache.org/FlumeUserGuide.html#memory-channel

I would recommend going through the same tutorial that you found here:

http://hortonworks.com/hadoop-tutorial/how-to-refine-and-visualize-server-log-data/

Within that tutorial, configure a memory channel and try starting it from the command line. Let me know if you have any issues along the way and I'll be glad to assist.

Thanks,

Rich Raposa

Certification Manager

[#8]

@rich This is perfect for an article..Thanks , Rich! @Daniel Hendrix