Support Questions
Find answers, ask questions, and share your expertise

Spark SQL on Hue notebook not executing properly

New Contributor

Hi All, Currently I am working on an ETL project, which uses spark largely for all of its computations.


For a POC I have tried integrating Spark with Hue and the integration worked fine.


But when I tried spark sql on the Hue notebook, the queries are executing fine, but it is executing in local mode always.

Another issue noticed is that the, the context is always running (Example : /apps/spark/local-1509061166070.inprogress).


The spark sql is connected to the spark thrift server to to access hive.


I am using Hue 3.12 and livy 3.12.