Support Questions

Find answers, ask questions, and share your expertise
Celebrating as our community reaches 100,000 members! Thank you!

Spark-shell keeps getting stuck!

New Contributor

I am currently going through this guide on ORC format table in Hive via Spark,

I have been having a lot of issues with the VMWare HDP Sandbox since the start, which I assume is because I am running it with only 2 CPUs and 22GB of RAM allocated.

The spark-shell command did execute fully once, but never after that till date. I have tried setting ZooKeeper, Oozie, and Ranger to maintanence mode to allow for some space to run, but this strategy is no longer working,

Spark-shell command is always stuck after the log level setting prompt. I have ensured that YARN server is running, but to no avail.

Any help is appreciated.


Master Collaborator

Have you tried to restart the cluster and cluster services. I think you need to keep all services are running because either directly or indirectly one service is dependent on other service.