Incrementally Loading flat files into hadoop (comma delimited text file)

[#1]

Hi All,

Hi bhagan,

@bhagan

Is it possible to load incremental text files (comma delimited text file) into hadoop as the data is increasing every minute. If we have folder containing date wise flat files in the local system, is it possible to load the files automatically into hadoop periodically.

If a particular file is loaded into hdfs, then it should load from the next record onwards.

What are the suitable ecosystems we can use to load the flat files.

Please do the needful.

.

[#2 · Accepted solution]

@Magesh Kumar

I believe this question is identical to another one you asked: https://community.hortonworks.com/questions/63947/incremental-flat-file-data-loading-into-hadoop.htm... If there are differences, please elaborate.