One late reply here: this bug fix may be relevant to the original problem:   https://github.com/cloudera/oryx/issues/99   I'll put this out soon in 1.0.1 ... View more

Hadoop still has config files for sure. They can end up wherever you want them to. I though they're still at $HADOOP_HOME/conf in the vanilla Hadoop tarball, but I took a look at 2.5.2 and it's at$HADOOP_HOME/etc/hadoop in fact. In any event if they're at /usr/local/hadoop/etc/hadoop in your installation, then that's what you set $HADOOP_CONF_DIR to. Just wherever they really are. This is one of Hadoop's standard environment variables. If you're up and running then this is working.   Yes that sounds like about what you do to install snappy. They are libs that should be present on the cluster machines. ... View more

Thanks veru much, I am new to Spark and drank the coolade on the communative and assoviative mandate for Cluster-based algorighms   I very much appreciate you providing me an accurate view on implementations.     I am very interested in the parelization of SNA and ML algorithms on clusters and appreciate any reading/references you can provide me.     Thanks again for your time and insignt and I appreciate any further insight you can provide.   In short; thanks Mate!   Chris ... View more

It looks like you asked for more resources than you configured YARN to offer, so check how much you can allocate in YARN and how much Spark asked for. I don't know about the ERROR; it may be a red herring. Please have a look at http://spark.apache.org/docs/latest/ for pretty good Spark docs. ... View more

Apologies, I'm mixing up 1.x and 2.x. The default evaluation metric in 1.x is mean average precision, or MAP. This is a measure of how much the top recommendations contained some items that were held out for the user. In local mode you can find lines like "Mean average precision: xxx" in the logs.   In distributed mode, now that I review the code, I don't see that it is ever logged. It is written to a file called "MAP" under the subdirectory for the iteration. I can make the mapper workers output their own local value of MAP at least.   In 2.x the metric is AUC, which is basically a measure of how likely it is that a 'good' recommendation (from the held out data set) ranks above a random item. It is a broader, different measure. This you should find printed in the logs if you're using 2.x for sure, along with hyperparams that yielded it. ... View more

hello  srowen,   what I meant by analysis using Mahout algorithms on top of hadoop cluster as well MapReduce preprocessing tasks for instnstance tokenization, stemming and translation of 10 million of tweets.    thank you      ... View more

Including the all the jars worked like a charm.  Regarding the "64MB limit",  I wasn't able to upload the uber-jar to HDFS via Hue (Error:Undefined) and from some searches I saw people claiming that they thought it was a size issue.   Thanks! ... View more