Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

analyzing streaming flat files into hdfs

analyzing streaming flat files into hdfs

New Contributor

i've been able to successfully load data into hdfs and run queries using hive now. But now I want to understand how one can create a metadata structure for a log file (which I know how to do) but then be able to add additional log files of the same format and not have to change the script.

 

so for example, if i have server log files that get generated hourly, can i just load the files into hdfs and the not change the hive scripts and still be able to analyze the incoming data as it comes in?