03-20-2019 03:55 AM - last edited on 03-20-2019 05:56 AM by cjervis
After upgrading Cloudera to 6.1
seems to be no longer available in the
hadoop jar /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/lib/solr/contrib/mr/search-mr-job.jar org.apache.solr.hadoop.HdfsFindTool WARNING: Use "yarn jar" to launch YARN applications. Exception in thread "main" java.lang.ClassNotFoundException: org.apache.solr.hadoop.HdfsFindTool at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.hadoop.util.RunJar.run(RunJar.java:306) at org.apache.hadoop.util.RunJar.main(RunJar.java:227)
are there any alternatives? Is there a way to run older version together with the 6.1 and newer stack? is this class contained in any other libraries?
03-21-2019 05:48 PM
03-22-2019 02:35 AM
The problem is, find doesn't seem to take the parameters that old tool, or OS level find accepts - and I cannot find documentation of viable expressions and its syntax:
hdfs dfs -find /example/path -mtime +14 find: Unexpected argument: -mtime
03-25-2019 04:51 AM
So - if I am correct, native hdfs find accepts only two expressions:
The following primary expressions are recognised: -name pattern -iname pattern Evaluates as true if the basename of the file matches the pattern using standard file system globbing. If -iname is used then the match is case insensitive. -print -print0 Always evaluates to true. Causes the current pathname to be written to standard output. If the -print0 expression is used then an ASCII NULL character is appended.
Which makes it useless for searching files-older-then - which was main use case for the HdfsFindTool. Is there any chance of the HdfsFindTool being brought back? Or any workaround for how to make it work with newer cloudera?