- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Loading data into HDFS from local filesystem, Flume or HDFS put?
- Labels:
-
Apache Flume
-
Apache Hadoop
Created ‎11-20-2017 06:00 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I want to load data(~300 GB) from local filesystem to HDFS. And i will be doing similar activity once every month.
What would be the feasible way to get this done. I am looking at Flume & HDFS Put options.
These are some files (XML) and not log data. I dont need any conversion, its a straight copy to HDFS.
Created ‎11-20-2017 06:33 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My personal preference is HDFS put over flume if those are the options. Even better would be HDF, but it sounds like a simple HDFS put would solve it
Created ‎11-20-2017 06:33 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My personal preference is HDFS put over flume if those are the options. Even better would be HDF, but it sounds like a simple HDFS put would solve it
