Support Questions
Find answers, ask questions, and share your expertise

hdfs file exists check

Hi, I am performing a basic check to see if a file exists in HDFS or not. I am using hdfs dos -test command for the same. But it doesn't seem to work correctly. In the documentation its mentioned as it would return 0 if the file exists. But I am not getting any output when the command is run. Let me know what need to be done to get this working. Please see the screenshot attached Thanks


virtualbox-hortonworks-sandbox-with-hdp-2png.jpg
1 ACCEPTED SOLUTION

Accepted Solutions

Contributor

@bigdata.neophyte The hdfs command doesn't print the result but sets it's return code to the shell. You'll need to test the return code from the 'hdfs dfs -test' command. On Linux try this:

hdfs dfs -test -e /tmp
echo $?
hdfs dfs -test -e /this_doesnt_exist
echo $?

View solution in original post

4 REPLIES 4

Contributor

@bigdata.neophyte The hdfs command doesn't print the result but sets it's return code to the shell. You'll need to test the return code from the 'hdfs dfs -test' command. On Linux try this:

hdfs dfs -test -e /tmp
echo $?
hdfs dfs -test -e /this_doesnt_exist
echo $?

View solution in original post

@bigdata.neophyte Here is the usage -

Usage: hadoop fs -test -[defsz] URI

Options:

  • -d: f the path is a directory, return 0.
  • -e: if the path exists, return 0.
  • -f: if the path is a file, return 0.
  • -s: if the path is not empty, return 0.
  • -z: if the file is zero length, return 0.

Example:

  • hadoop fs -test -e filename

Check - http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#test

New Contributor

Hi Team , Is there any way to test multiple files exist or not in single command?? like below.. pls suggest

  • hadoop fs -test -e filename1 filename2