Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Loading data into HDFS from local filesystem, Flume or HDFS put?

avatar
Visitor

I want to load data(~300 GB) from local filesystem to HDFS. And i will be doing similar activity once every month.

What would be the feasible way to get this done. I am looking at Flume & HDFS Put options.

These are some files (XML) and not log data. I dont need any conversion, its a straight copy to HDFS.

1 ACCEPTED SOLUTION

avatar
Guru

My personal preference is HDFS put over flume if those are the options. Even better would be HDF, but it sounds like a simple HDFS put would solve it

View solution in original post

1 REPLY 1

avatar
Guru

My personal preference is HDFS put over flume if those are the options. Even better would be HDF, but it sounds like a simple HDFS put would solve it