Web interface for querying Spark SQL ?

[#1]

I am trying to figure out a tool for querying spark SQL queries and provides features like Hue (user authentication , Download result and restrictions to access tables)

Zeppelin and Jupyter are more like a notebooks and doesn't have features like user authentication and download result feature.

Any suggestions around this ?

Thanks in advance.

[#2 · Accepted solution]

I thought that on a secure cluster zeppelin can only make queries as the user hosting the web ui...though I'm not sure there.

Spark SQL doesn't do user authentication in general, not via the thrift server (JBDC and especially ODBC). Nor does it do column-level access control as Hive does. It's just going straight at the files themselves. So it's not that locked down.

[#3]

Agree @stevel there will not be any user authentication in Spark SQL .

I was trying to figure out a tool for firing spark SQL queries and provides features like Hue.

Hue internally supports user authentication and interface for hive , pig queries.

[#4]

Hue has something behind the scenes called Livy, which is a little REST server doing the work...they haven't teased that out and made it standalone which is a shame. There's actually something very interested starting in the apache incubator, IBM's Spark Kernel code (which will be renamed during the incubation process)..this lets you wire up Jupyter directly, but also offers the ability to upload code callbacks into the spark cluster itself. I think that's pretty nice, and will be keeping an eye on it —though I don't know when it will be ready for broad use.

[#5]

Thanks @stevel

[#6]

@vshukla mentioned that LDAP authentication for Zeppelin is planned for the next techpreview (currently slated for Jan)

[#7]

@Sandeep Nemuri can you accept the best answer to close this thread?

[#8]

The latest technical preview for Zeppelin and HDP 2.4 was announced here: http://hortonworks.com/blog/announcing-ga-of-apache-spark-1-6-in-hortonworks-data-platformtm-hdp2-4/

Stayed tuned for our final Zeppelin technical preview which will work on a secure cluster.