Support Questions
Find answers, ask questions, and share your expertise

Hadoop filesystem Commands

What does hadoop fs -test do?

What are the other set of options which can be used along with -test like hadoop fs -test -d.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Hadoop filesystem Commands

Mentor

@Bala Vignesh N V

hadoop fs has been deprecated, From hadoop 2.0 use hdfs dfs instead.Having said that hadoop fs -test -[defsz] URI is used to check the hdfs and it takes the below options:

-d: f the path is a directory, return 0. 
-e: if the path exists, return 0. 
-f: if the path is a file, return 0. 
-s: if the path is not empty, return 0. 
-z: if the file is zero length, return 0.

Example

Check for valid directories in hdfs

$ hdfs dfs -ls /
Found 7 items
drwxrwx---   - ambari-qa hdfs          0 2017-10-19 14:13 /user/ambari-qa
drwxr-xr-x   - druid     hdfs          0 2017-10-19 20:49 /user/druid
drwxr-xr-x   - hbase     hdfs          0 2017-10-19 13:43 /user/hbase
drwxr-xr-x   - hcat      hdfs          0 2017-10-19 13:53 /user/hcat
drwxr-xr-x   - hive      hdfs          0 2017-10-19 13:53 /user/hive
drwxrwxr-x   - oozie     hdfs          0 2017-10-19 13:57 /user/oozie
drwxr-xr-x   - zeppelin  hdfs          0 2017-10-19 19:25 /user/zeppelin 

Test whether path exists,the below returns 0

$ hdfs dfs -test -e /user/druid

Hope that helps

View solution in original post

3 REPLIES 3

Re: Hadoop filesystem Commands

Mentor

@Bala Vignesh N V

hadoop fs has been deprecated, From hadoop 2.0 use hdfs dfs instead.Having said that hadoop fs -test -[defsz] URI is used to check the hdfs and it takes the below options:

-d: f the path is a directory, return 0. 
-e: if the path exists, return 0. 
-f: if the path is a file, return 0. 
-s: if the path is not empty, return 0. 
-z: if the file is zero length, return 0.

Example

Check for valid directories in hdfs

$ hdfs dfs -ls /
Found 7 items
drwxrwx---   - ambari-qa hdfs          0 2017-10-19 14:13 /user/ambari-qa
drwxr-xr-x   - druid     hdfs          0 2017-10-19 20:49 /user/druid
drwxr-xr-x   - hbase     hdfs          0 2017-10-19 13:43 /user/hbase
drwxr-xr-x   - hcat      hdfs          0 2017-10-19 13:53 /user/hcat
drwxr-xr-x   - hive      hdfs          0 2017-10-19 13:53 /user/hive
drwxrwxr-x   - oozie     hdfs          0 2017-10-19 13:57 /user/oozie
drwxr-xr-x   - zeppelin  hdfs          0 2017-10-19 19:25 /user/zeppelin 

Test whether path exists,the below returns 0

$ hdfs dfs -test -e /user/druid

Hope that helps

View solution in original post

Re: Hadoop filesystem Commands

Thanks @Geoffrey Shelton Okot. This is what I'm looking for.

Re: Hadoop filesystem Commands

Expert Contributor