Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

HDFSFindTool exec parameters

HDFSFindTool exec parameters

New Contributor

Anyone used HdfsFindTool? I cant figure out how the -exec works

I want to find files older than X days and delete them.(but first i wanted to test with ls)

I've tried hdfs dfs -ls, dfs -ls, fs -ls, -ls

I keep getting NullPointerException

 

# hadoop jar search-mr-1.3.0-job.jar org.apache.solr.hadoop.HdfsFindTool -find hdfs:///tmp/ -type f -mtime +215 -exec -ls {} \;
-find: Fatal internal error
java.lang.NullPointerException
    at org.apache.hadoop.fs.shell.find.Exec.initialise(Exec.java:109)
    at org.apache.hadoop.fs.shell.find.BaseExpression.initialise(BaseExpression.java:64)
    at org.apache.hadoop.fs.shell.Find.processArguments(Find.java:383)
    at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:190)
    at org.apache.hadoop.fs.shell.Command.run(Command.java:154)
    at org.apache.hadoop.fs.FsShell.run(FsShell.java:254)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at org.apache.solr.hadoop.HdfsFindTool.main(HdfsFindTool.java:43)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:622)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

 

 

I've tried both search-mr-1.3.0-job.jar and search-mr-1.3.0.jar, don't really notice any difference.

How do I use the -exec function to remove files that -find finds?

 

2 REPLIES 2

Re: HDFSFindTool exec parameters

New Contributor

Hi Raboo.

 

Could you try with this?

 

files=$(hadoop jar search-mr-1.3.0-job.jar org.apache.solr.hadoop.HdfsFindTool -find hdfs:///tmp/ -type f -mtime +215)
hadoop fs -rm $files

Regards.

Highlighted

Re: HDFSFindTool exec parameters

New Contributor

I too am having trouble using the -exec option. The find functionality seems to work fine for me, but getting it to execute simple commands like rm or ls seems problematic. Is there a special syntax which must  be used?