Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hdfs file exists check

Solved Go to solution
Highlighted

hdfs file exists check

New Contributor

Hi, I am performing a basic check to see if a file exists in HDFS or not. I am using hdfs dos -test command for the same. But it doesn't seem to work correctly. In the documentation its mentioned as it would return 0 if the file exists. But I am not getting any output when the command is run. Let me know what need to be done to get this working. Please see the screenshot attached Thanks


virtualbox-hortonworks-sandbox-with-hdp-2png.jpg
1 ACCEPTED SOLUTION

Accepted Solutions

Re: hdfs file exists check

Contributor

@bigdata.neophyte The hdfs command doesn't print the result but sets it's return code to the shell. You'll need to test the return code from the 'hdfs dfs -test' command. On Linux try this:

hdfs dfs -test -e /tmp
echo $?
hdfs dfs -test -e /this_doesnt_exist
echo $?
4 REPLIES 4

Re: hdfs file exists check

Contributor

@bigdata.neophyte The hdfs command doesn't print the result but sets it's return code to the shell. You'll need to test the return code from the 'hdfs dfs -test' command. On Linux try this:

hdfs dfs -test -e /tmp
echo $?
hdfs dfs -test -e /this_doesnt_exist
echo $?

Re: hdfs file exists check

New Contributor

Re: hdfs file exists check

@bigdata.neophyte Here is the usage -

Usage: hadoop fs -test -[defsz] URI

Options:

  • -d: f the path is a directory, return 0.
  • -e: if the path exists, return 0.
  • -f: if the path is a file, return 0.
  • -s: if the path is not empty, return 0.
  • -z: if the file is zero length, return 0.

Example:

  • hadoop fs -test -e filename

Check - http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#test

Re: hdfs file exists check

New Contributor

Hi Team , Is there any way to test multiple files exist or not in single command?? like below.. pls suggest

  • hadoop fs -test -e filename1 filename2