We don't support Hive on Spark and support Hive on Tez or SparkSQL for SQL solutions in Hadoop. Let' take it offline for customer input. ... View more

Ambari needs headless Keytab so it can start services without prompting for password. That's how Spark's History server is started (i.e. w/o prompting for password) When an end user is submitting a spark job, they can use either a headless keytab or type the kerberos password. ... View more

We have an experimental Spark HBase connector, https://github.com/zhzhan/shc With the following features First class support for DataFrame API JSON based catalog with rich data type support Performant, scalable, enterprise-ready Partition Pruning Predicate Pushdown Scan optimizations Data Locality Composite Rowkey Leverage existing work in the HBase community Please take look at the README of the above project. Also see example https://github.com/zhzhan/shc/blob/master/src/main/scala/org/apache/spark/sql/execution/datasources/hbase/examples/HBaseSource.scala ... View more

Iobna, Every vendor ships Apache Spark, so there isn't a difference there. But there are other differences you can use to evaluate which vendor you choose As Neeraj mention the difference between the Vendor themselves, all open source or partial open sources or limited open sources. Which version of Spark is supported in each vendor's distro: HDP with 2.3.4 coming out (this week) supports Spark 1.5.2 Which component of Spark is supported by the vendor (we support SparkCore, ML, Streaming, SQL) What is the focus on Spark for each vendor (ours is detailed here http://hortonworks.com/blog/spark-hdp-perfect-tog... How well the vendor can support you (Hortonwork's support is top rated) There are many other factors to consider, but this should give you some ideas. Thanks, Vinay ... View more