Sandbox - ls command error - new to Linux and this.

[#1]

Hi,

I wondered if someone could give me some pointers as to where I'm going wrong. I've spun up the Azure HortonWorks HDP 2.4 machine and I can connect to it via Putty and WINSCP. My issue however is with running some of the hdfs dfs commands however (demo'd in a PluralSight video').

This is what I'm doing;

1. Open Putty on my machine and connect using my login.

2. Then I run "hdfs dfs -ls" - I get back the message "ls: '.':No such file or directory".

3. Looking at hdfs dfs -help's definition of -ls it says "if the path is not specified" which its not then "the contents of /user/<currentuser? will be listed",

4. So then I tried "hdfs dfs - ls /user" and I get back some results, although there is no folder matching my login name.

Am I missing something obvious or is my user folder a little bit messed up for some reason? if I just run "ls" from the prompt I get no results. If I browser to the /usr folder I can see other folders but not one with my name.

Any help/ideas would be greatly appreciated.

Thanks,

Nic

[#2 · Accepted solution]

create your user directory

sudo -u hdfs hdfs dfs -mkdir /user/username
sudo -u hdfs hdfs dfs -chown -R hdfs:username /user/username

[#3]

Hi Artem,

Thanks for reply, when I run the first command via Putty I get told

"mkdir: Cannot create directory /user/nic. Name node is in safe mode." I'm not sure why it would be in safe mode unless that the default.

Thanks,

Nic

[#4]

Run the following command, then repeat the earlier commands

sudo -u hdfs hdfs dfsadmin -safemode leave

[#5]

Hi Artem,

Firstly thanks for your time with this. So I ran the command to have HDFS leave safe mode, worked fine.

Then I ran the command to create my hdfs folder and grant permissions, worked fine.

so now when I run "hdfs dfs -ls" I don't get the error, instead it thinks about it for a minute and then returns nothing and awaits my next command, not sure if that's right or not. If I run "hdfs dfs -ls /user/" then again it thinks about it then returns a list of folders (including me) so this is good, matches up to the PluralSight video, except he has the hue folder and I don't.

But for now I'm back in the game, so thanks and its nice to know what the commands actually do as well.

Thanks,

Nic

[#6]

@Nic Hopper

yes so now you're good to go as you have access to HDFS via CLI (Command Shell). You can now upload and download files using CLI

sudo -u username hdfs dfs -put file /user/username/
sudo -u username hdfs dfs -get file /user/username/file

if you have your username available locally on the OS then you can skip the part with

sudo -u username

just change to your userID and run those commands

su username
hdfs dfs -ls
hdfs dfs -put
hdfs dfs -get

also as a good practice, please accept the answer to close this thread.

[#7]

Thanks again Artem for your response and anyone else who took the time read it.