I wondered if someone could give me some pointers as to where I'm going wrong. I've spun up the Azure HortonWorks HDP 2.4 machine and I can connect to it via Putty and WINSCP. My issue however is with running some of the hdfs dfs commands however (demo'd in a PluralSight video').
This is what I'm doing;
1. Open Putty on my machine and connect using my login.
2. Then I run "hdfs dfs -ls" - I get back the message "ls: '.':No such file or directory".
3. Looking at hdfs dfs -help's definition of -ls it says "if the path is not specified" which its not then "the contents of /user/<currentuser? will be listed",
4. So then I tried "hdfs dfs - ls /user" and I get back some results, although there is no folder matching my login name.
Am I missing something obvious or is my user folder a little bit messed up for some reason? if I just run "ls" from the prompt I get no results. If I browser to the /usr folder I can see other folders but not one with my name.
Any help/ideas would be greatly appreciated.
Thanks for reply, when I run the first command via Putty I get told
"mkdir: Cannot create directory /user/nic. Name node is in safe mode." I'm not sure why it would be in safe mode unless that the default.
Firstly thanks for your time with this. So I ran the command to have HDFS leave safe mode, worked fine.
Then I ran the command to create my hdfs folder and grant permissions, worked fine.
so now when I run "hdfs dfs -ls" I don't get the error, instead it thinks about it for a minute and then returns nothing and awaits my next command, not sure if that's right or not. If I run "hdfs dfs -ls /user/" then again it thinks about it then returns a list of folders (including me) so this is good, matches up to the PluralSight video, except he has the hue folder and I don't.
But for now I'm back in the game, so thanks and its nice to know what the commands actually do as well.
yes so now you're good to go as you have access to HDFS via CLI (Command Shell). You can now upload and download files using CLI
sudo -u username hdfs dfs -put file /user/username/ sudo -u username hdfs dfs -get file /user/username/file
if you have your username available locally on the OS then you can skip the part with
sudo -u username
just change to your userID and run those commands
su username hdfs dfs -ls hdfs dfs -put hdfs dfs -get
also as a good practice, please accept the answer to close this thread.