That's perhaps too broad to answer here. Generally, any algorithm that is data-parallel will do well on Spark (or indeed, on MapReduce). And ones that aren't data-parallel do not. I am not familiar with any of those algorithms, but that's the question to answer. ... View more

Your local Hive CLI JVM heap size is insufficient for even building and submitting the job. Please try raising it as below, and retrying: ~> export HADOOP_CLIENT_OPTS="-Xmx2g" ~> hive -e "select count(station_id) from aws_new;" ... View more

You receive the error because the 'hbase' user does not have a login shell assigned to it. You can set a shell for the 'hbase' user on the machine, to allow direct 'su' based login to that user, by following http://www.cyberciti.biz/faq/howto-set-bash-as-your-default-shell/ However, if your goal is to simply use the 'hbase' user for running superuser level commands, we instead recommend using 'sudo' style commands. For example: ~> sudo -u hbase hbase hbck ~> sudo -u hbase hbase shell You can also invoke a shell as the 'hbase' user in certain cases, via: ~> sudo -u hbase /bin/bash ... View more