Reply
New Contributor
Posts: 1
Registered: ‎11-08-2014

HDFSFindTool exec parameters

Anyone used HdfsFindTool? I cant figure out how the -exec works

I want to find files older than X days and delete them.(but first i wanted to test with ls)

I've tried hdfs dfs -ls, dfs -ls, fs -ls, -ls

I keep getting NullPointerException

 

# hadoop jar search-mr-1.3.0-job.jar org.apache.solr.hadoop.HdfsFindTool -find hdfs:///tmp/ -type f -mtime +215 -exec -ls {} \;
-find: Fatal internal error
java.lang.NullPointerException
    at org.apache.hadoop.fs.shell.find.Exec.initialise(Exec.java:109)
    at org.apache.hadoop.fs.shell.find.BaseExpression.initialise(BaseExpression.java:64)
    at org.apache.hadoop.fs.shell.Find.processArguments(Find.java:383)
    at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:190)
    at org.apache.hadoop.fs.shell.Command.run(Command.java:154)
    at org.apache.hadoop.fs.FsShell.run(FsShell.java:254)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at org.apache.solr.hadoop.HdfsFindTool.main(HdfsFindTool.java:43)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:622)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

 

 

I've tried both search-mr-1.3.0-job.jar and search-mr-1.3.0.jar, don't really notice any difference.

How do I use the -exec function to remove files that -find finds?

 

New Contributor
Posts: 4
Registered: ‎01-19-2016

Re: HDFSFindTool exec parameters

Hi Raboo.

 

Could you try with this?

 

files=$(hadoop jar search-mr-1.3.0-job.jar org.apache.solr.hadoop.HdfsFindTool -find hdfs:///tmp/ -type f -mtime +215)
hadoop fs -rm $files

Regards.

New Contributor
Posts: 1
Registered: ‎12-19-2017

Re: HDFSFindTool exec parameters

I too am having trouble using the -exec option. The find functionality seems to work fine for me, but getting it to execute simple commands like rm or ls seems problematic. Is there a special syntax which must  be used?

Announcements