Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Not able find directory created under HDFS

SOLVED Go to solution

Not able find directory created under HDFS

New Contributor

Hi Friends, I am trying to understand the PIG scripting.

I have created directory called pig_demo and am not able to find when i do hadoop fs -ls /

Appericiate your timely response on this.

 

details are given bleow

1) [cloudera@quickstart ~]$ pwd
/home/cloudera
[cloudera@quickstart ~]$ hadoop fs -ls /
Found 5 items
drwxr-xr-x   - hbase supergroup          0 2015-10-29 13:01 /hbase
drwxr-xr-x   - solr  solr                0 2015-06-09 08:55 /solr
drwxrwxrwx   - hdfs  supergroup          0 2015-10-09 06:24 /tmp
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:56 /user
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:54 /var

 

2) [cloudera@quickstart ~]$ hadoop fs -mkdir pig_demo
mkdir: `pig_demo': File exists
[cloudera@quickstart ~]$ hadoop fs -ls /
Found 5 items
drwxr-xr-x   - hbase supergroup          0 2015-10-29 13:01 /hbase
drwxr-xr-x   - solr  solr                0 2015-06-09 08:55 /solr
drwxrwxrwx   - hdfs  supergroup          0 2015-10-09 06:24 /tmp
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:56 /user
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:54 /var

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Not able find directory created under HDFS

Master Collaborator

Also note that when you just run 'ls' or 'pwd', that's referring to your local / native filesystem. What directory you are currently in in your shell has no effect on what directory it shows you when you say 'hdfs dfs -ls'. Now, there's a /usr directory on the local / native filesystem, but that's a complete separate idea from the /user directory in HDFS.

6 REPLIES 6

Re: Not able find directory created under HDFS

Explorer

Have you cheked in /user directory?

Re: Not able find directory created under HDFS

Master Collaborator

Check in /user/cloudera. Unless you're the 'hdfs' user, HDFS treats
/user/[username] as your home directory. When your paths don't start with
/, they go in there. If they DO start with /, it doesn't resolve it
relative to your home directory. If you just type 'hadoop fs -ls', it'll
look in your home directory and you should see it.

Re: Not able find directory created under HDFS

New Contributor

I am not able to find under usr

[cloudera@quickstart usr]$ ls
bin  games    jars  lib    libexec  sbin   src
etc  include  java  lib64  local    share  tmp

[cloudera@quickstart usr]$

 

[cloudera@quickstart /]$ hadoop fs -ls /
Found 5 items
drwxr-xr-x   - hbase supergroup          0 2015-10-29 13:01 /hbase
drwxr-xr-x   - solr  solr                0 2015-06-09 08:55 /solr
drwxrwxrwx   - hdfs  supergroup          0 2015-10-09 06:24 /tmp
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:56 /user
drwxr-xr-x   - hdfs  supergroup          0 2015-06-09 08:54 /var

Highlighted

Re: Not able find directory created under HDFS

Rising Star

I think what the earlier responders were suggesting is that you type this at the Linux command prompt:

 

hdfs dfs -ls /user/cloudera

 

You should see your new directory, pig-demo, listed as a subdirectory of /user/cloudera.

Can you try this?

 

(Note: you can use 'hadoop fs -ls /user/cloudera' too, but 'hadoop fs' is being deprecated so best to switch to using 'hdfs dfs ...')

 

Re: Not able find directory created under HDFS

Master Collaborator

Also note that when you just run 'ls' or 'pwd', that's referring to your local / native filesystem. What directory you are currently in in your shell has no effect on what directory it shows you when you say 'hdfs dfs -ls'. Now, there's a /usr directory on the local / native filesystem, but that's a complete separate idea from the /user directory in HDFS.

Re: Not able find directory created under HDFS

New Contributor

Sean , Thank you very much for providing quick resolution