question Re: Hadoop filesystem Commands in Archives of Support Questions (Read Only)

Hadoop filesystem Commands

balavignesh_nag — Thu, 16 Nov 2017 19:23:27 GMT

What does hadoop fs -test do?

What are the other set of options which can be used along with -test like hadoop fs -test -d.

Re: Hadoop filesystem Commands

Shelton — Thu, 16 Nov 2017 20:12:18 GMT

hadoop fs has been deprecated, From hadoop 2.0 use hdfs dfs instead.Having said that hadoop fs -test -[defsz] URI is used to check the hdfs and it takes the below options:

-d: f the path is a directory, return 0. 
-e: if the path exists, return 0. 
-f: if the path is a file, return 0. 
-s: if the path is not empty, return 0. 
-z: if the file is zero length, return 0.

Example

Check for valid directories in hdfs

$ hdfs dfs -ls /
Found 7 items
drwxrwx---   - ambari-qa hdfs          0 2017-10-19 14:13 /user/ambari-qa
drwxr-xr-x   - druid     hdfs          0 2017-10-19 20:49 /user/druid
drwxr-xr-x   - hbase     hdfs          0 2017-10-19 13:43 /user/hbase
drwxr-xr-x   - hcat      hdfs          0 2017-10-19 13:53 /user/hcat
drwxr-xr-x   - hive      hdfs          0 2017-10-19 13:53 /user/hive
drwxrwxr-x   - oozie     hdfs          0 2017-10-19 13:57 /user/oozie
drwxr-xr-x   - zeppelin  hdfs          0 2017-10-19 19:25 /user/zeppelin

Test whether path exists,the below returns 0

$ hdfs dfs -test -e /user/druid

Hope that helps

Re: Hadoop filesystem Commands

dheer_vijji_rag — Thu, 16 Nov 2017 20:15:47 GMT

@Bala Vignesh N V

this can help you: https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-common/FileSystemShell.html#test

Re: Hadoop filesystem Commands

balavignesh_nag — Thu, 16 Nov 2017 21:13:23 GMT

Thanks @Geoffrey Shelton Okot. This is what I'm looking for.