Support Questions

Find answers, ask questions, and share your expertise

Incrementally Loading flat files into hadoop (comma delimited text file)

avatar

Hi All,

Hi bhagan,

@bhagan

Is it possible to load incremental text files (comma delimited text file) into hadoop as the data is increasing every minute. If we have folder containing date wise flat files in the local system, is it possible to load the files automatically into hadoop periodically.

If a particular file is loaded into hdfs, then it should load from the next record onwards.

What are the suitable ecosystems we can use to load the flat files.

Please do the needful.

.

1 ACCEPTED SOLUTION

avatar
Guru
@Magesh Kumar

I believe this question is identical to another one you asked: https://community.hortonworks.com/questions/63947/incremental-flat-file-data-loading-into-hadoop.htm... If there are differences, please elaborate.

View solution in original post

1 REPLY 1

avatar
Guru
@Magesh Kumar

I believe this question is identical to another one you asked: https://community.hortonworks.com/questions/63947/incremental-flat-file-data-loading-into-hadoop.htm... If there are differences, please elaborate.