Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Web interface for querying Spark SQL ?

avatar
Explorer

I am trying to figure out a tool for querying spark SQL queries and provides features like Hue (user authentication , Download result and restrictions to access tables)

Zeppelin and Jupyter are more like a notebooks and doesn't have features like user authentication and download result feature.

Any suggestions around this ?

Thanks in advance.

1 ACCEPTED SOLUTION

avatar

I thought that on a secure cluster zeppelin can only make queries as the user hosting the web ui...though I'm not sure there.

Spark SQL doesn't do user authentication in general, not via the thrift server (JBDC and especially ODBC). Nor does it do column-level access control as Hive does. It's just going straight at the files themselves. So it's not that locked down.

View solution in original post

7 REPLIES 7

avatar

I thought that on a secure cluster zeppelin can only make queries as the user hosting the web ui...though I'm not sure there.

Spark SQL doesn't do user authentication in general, not via the thrift server (JBDC and especially ODBC). Nor does it do column-level access control as Hive does. It's just going straight at the files themselves. So it's not that locked down.

avatar
Explorer

Agree @stevel there will not be any user authentication in Spark SQL .

I was trying to figure out a tool for firing spark SQL queries and provides features like Hue.

Hue internally supports user authentication and interface for hive , pig queries.

avatar

Hue has something behind the scenes called Livy, which is a little REST server doing the work...they haven't teased that out and made it standalone which is a shame. There's actually something very interested starting in the apache incubator, IBM's Spark Kernel code (which will be renamed during the incubation process)..this lets you wire up Jupyter directly, but also offers the ability to upload code callbacks into the spark cluster itself. I think that's pretty nice, and will be keeping an eye on it —though I don't know when it will be ready for broad use.

avatar
Explorer

Thanks @stevel

avatar

@vshukla mentioned that LDAP authentication for Zeppelin is planned for the next techpreview (currently slated for Jan)

avatar
Master Mentor

@Sandeep Nemuri can you accept the best answer to close this thread?

avatar
Contributor

The latest technical preview for Zeppelin and HDP 2.4 was announced here: http://hortonworks.com/blog/announcing-ga-of-apache-spark-1-6-in-hortonworks-data-platformtm-hdp2-4/

Stayed tuned for our final Zeppelin technical preview which will work on a secure cluster.