New Contributor
Posts: 3
Registered: ‎01-05-2018

Spark Cluster - Difficulty in getting spark sessions at the same time.

I have a single node cluster that is being configured with Cloudera distribution CDH/Spark. Whey my application is submitting a job to the spark cluster (via Livy) the execution starts and goes on for 15 mins. What I have realized is that, during this duration of 15 mins if I open a spark-shell session then I don't get handle of the spark session. I guess, multiple users are not able to get resources from spark cluster. Could you please suggest what needs to be done in order to get rid of this issue.