Apache Flume and parquet

[#1]

Hi.

Is it possible configure Apache Flume to save my logs in HDFS with Parquet?

Thanks very much!!!!

Miguel Angel.

[#2]

Yes. Use Impala or Hive to convert to Parquet stream from Flume is a good option, although it would be nice to have it natively.

Thanks!!!!

Miguel Angel.

[#3]

You're welcome!

[#4]

Hi Mike,

How do you convert the avro data to parquet, and what do you use to schedule this process?

Is the code hosted somewhere? Thanks.

[#5]

Impala can do the conversion via SQL statements. I'd recommend asking the
Impala guys for advice there as my information is a bit dated on this
front, now that views and improved meta store features have been added.

Mike