10-27-2017 09:37 AM - last edited on 10-28-2017 06:22 AM by cjervis
Hi All, Currently I am working on an ETL project, which uses spark largely for all of its computations.
For a POC I have tried integrating Spark with Hue and the integration worked fine.
But when I tried spark sql on the Hue notebook, the queries are executing fine, but it is executing in local mode always.
Another issue noticed is that the, the context is always running (Example : /apps/spark/local-1509061166070.inprogress).
The spark sql is connected to the spark thrift server to to access hive.
I am using Hue 3.12 and livy 3.12.