- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Why can't I upload a simple text file onto HDFS ?
- Labels:
-
Apache Hadoop
Created ‎04-13-2017 08:10 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I've been trying to upload a basic txt file onto HDFS but it won't upload. I have tried multiple files of different sizes and extensions and found the reason to be the size of my text file (approx 150 bytes). All files over around 10kb seems to work fine... Does anyone have a solution to this ? Thanks
Kelvin
Created ‎04-13-2017 01:19 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Based on the screen shots you are providing, you are attempting to push the file to HDFS using the command line within the Sandbox itself. However, you are specifying a file path that is local to the computer running the VIrtualBox Sandbox VM. That won't work. The Sandbox has no way of knowing how to access "C:\".
You must first push the file to the Sandbox using WinSCP. Then you can use the hdfs dsfs -put command using a local directory within the Sandbox (something like /root/<my filename>).
Created ‎04-13-2017 01:57 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Copy your file from windows to the host 127.0.0.1. From local directory copy to HDFS using
hadoop fs -ls <local file path> <hdfs path>
Created ‎04-13-2017 01:19 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Based on the screen shots you are providing, you are attempting to push the file to HDFS using the command line within the Sandbox itself. However, you are specifying a file path that is local to the computer running the VIrtualBox Sandbox VM. That won't work. The Sandbox has no way of knowing how to access "C:\".
You must first push the file to the Sandbox using WinSCP. Then you can use the hdfs dsfs -put command using a local directory within the Sandbox (something like /root/<my filename>).
Created ‎04-13-2017 03:05 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks ! That worked like a charm. I still have no idea why it doesn't let me upload using the HDFS UI so if you know why then I would love to know.

- « Previous
-
- 1
- 2
- Next »