Reply
New Contributor
Posts: 5
Registered: ‎10-09-2015
Accepted Solution

Not able find directory created under HDFS

Hi Friends, I am trying to understand the PIG scripting.

I have created directory called pig_demo and am not able to find when i do hadoop fs -ls /

Appericiate your timely response on this.

 

details are given bleow

1) [cloudera@quickstart ~]$ pwd
/home/cloudera
[cloudera@quickstart ~]$ hadoop fs -ls /
Found 5 items
drwxr-xr-x   - hbase supergroup          0 2015-10-29 13:01 /hbase
drwxr-xr-x   - solr  solr                0 2015-06-09 08:55 /solr
drwxrwxrwx   - hdfs  supergroup          0 2015-10-09 06:24 /tmp
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:56 /user
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:54 /var

 

2) [cloudera@quickstart ~]$ hadoop fs -mkdir pig_demo
mkdir: `pig_demo': File exists
[cloudera@quickstart ~]$ hadoop fs -ls /
Found 5 items
drwxr-xr-x   - hbase supergroup          0 2015-10-29 13:01 /hbase
drwxr-xr-x   - solr  solr                0 2015-06-09 08:55 /solr
drwxrwxrwx   - hdfs  supergroup          0 2015-10-09 06:24 /tmp
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:56 /user
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:54 /var

Contributor
Posts: 30
Registered: ‎08-27-2015

Re: Not able find directory created under HDFS

[ Edited ]

Have you cheked in /user directory?

Cloudera Employee
Posts: 435
Registered: ‎07-12-2013

Re: Not able find directory created under HDFS

[ Edited ]

Check in /user/cloudera. Unless you're the 'hdfs' user, HDFS treats
/user/[username] as your home directory. When your paths don't start with
/, they go in there. If they DO start with /, it doesn't resolve it
relative to your home directory. If you just type 'hadoop fs -ls', it'll
look in your home directory and you should see it.

New Contributor
Posts: 5
Registered: ‎10-09-2015

Re: Not able find directory created under HDFS

I am not able to find under usr

[cloudera@quickstart usr]$ ls
bin  games    jars  lib    libexec  sbin   src
etc  include  java  lib64  local    share  tmp

[cloudera@quickstart usr]$

 

[cloudera@quickstart /]$ hadoop fs -ls /
Found 5 items
drwxr-xr-x   - hbase supergroup          0 2015-10-29 13:01 /hbase
drwxr-xr-x   - solr  solr                0 2015-06-09 08:55 /solr
drwxrwxrwx   - hdfs  supergroup          0 2015-10-09 06:24 /tmp
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:56 /user
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:54 /var

Highlighted
Cloudera Employee Sue
Cloudera Employee
Posts: 44
Registered: ‎09-11-2015

Re: Not able find directory created under HDFS

I think what the earlier responders were suggesting is that you type this at the Linux command prompt:

 

hdfs dfs -ls /user/cloudera

 

You should see your new directory, pig-demo, listed as a subdirectory of /user/cloudera.

Can you try this?

 

(Note: you can use 'hadoop fs -ls /user/cloudera' too, but 'hadoop fs' is being deprecated so best to switch to using 'hdfs dfs ...')

 

Cloudera Employee
Posts: 435
Registered: ‎07-12-2013

Re: Not able find directory created under HDFS

Also note that when you just run 'ls' or 'pwd', that's referring to your local / native filesystem. What directory you are currently in in your shell has no effect on what directory it shows you when you say 'hdfs dfs -ls'. Now, there's a /usr directory on the local / native filesystem, but that's a complete separate idea from the /user directory in HDFS.

New Contributor
Posts: 5
Registered: ‎10-09-2015

Re: Not able find directory created under HDFS

Sean , Thank you very much for providing quick resolution

Announcements