Thank you for the answer ... View more

Thank you for the advice,the new feature of windowing seem able to solve my problem but only concern is the in memory capability to hold 1 hour data , may i know any example for how to configure /do the disk cache ? Between i found out some example of doing append in hdfs http://stackoverflow.com/questions/32339602/append-to-file-in-hdfs-cdh-5-4-5 the CDH platform can do the append ? ... View more

Appreciate for the advice between the file name will be named something like ddmmyyyy-hh. I want to group the log by hourly and the event per second can be changed, so the number of tuples and file side cannot be determined. In this case how to do it? ... View more