The only thing you'll lose out if you do the fresh approach is the history, but not the functionality/correctness of the newly added jobs. I'd pick using the fresh mysql approach if it is an option, cause delving down into extracting data from Apache Derby DB for moving it onto MySQL is a tad more painful operation. ... View more

Ah I didn't notice JD's post when replying by email, my bad. ... View more

> * Is this available in CDH4.5.0 running with hbase 0.94.6? Yes. See https://github.com/cloudera/hbase/blob/cdh4.5.0-release/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java#L1507. > * Do i set this in the safety valvle for the regionserver settings? Yes, this goes into the Advanced properties (safety valve for hbase-site.xml) of the Region Servers, as thats where compactions of store files are performed. > * Can I verify this is the current policy running on major compactions? There's no direct way AFAICT, but setting the property should suffice in enabling that sub-routine, per the logic shown above. ... View more

Hi, If you use MR1, then the /usr/lib/hadoop-mapreduce will not be on your classpath, but /usr/lib/hadoop-0.20-mapreduce would. For MR1, these set of classes reside under /usr/lib/hadoop-0.20-mapreduce/contrib/streaming/*.jar, which is typically not on the default classpath. Can you try the below, perhaps? ~> export HADOOP_CLASSPATH="/usr/lib/hadoop-0.20-mapreduce/contrib/streaming/*" ~> hadoop jar your-jar... ... View more

Many thanks for following up! I do think the directory-or-file can be checked for before traversal, or at least the error message may be improved, for which I've filed https://issues.apache.org/jira/browse/HDFS-5802. The shell utils handle this in a clearer way btw: ➜ ~ ls foo/file ls: foo/file: Not a directory ➜ ~ hadoop fs -ls foo/file ls: `foobar/file': No such file or directory ... View more