Reply
Highlighted
New Contributor
Posts: 10
Registered: ‎08-25-2014

analyzing streaming flat files into hdfs

i've been able to successfully load data into hdfs and run queries using hive now. But now I want to understand how one can create a metadata structure for a log file (which I know how to do) but then be able to add additional log files of the same format and not have to change the script.

 

so for example, if i have server log files that get generated hourly, can i just load the files into hdfs and the not change the hive scripts and still be able to analyze the incoming data as it comes in?

Announcements
New solutions