Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Unnecessary daemons for spark-shell

avatar
New Contributor

I'm trying use latest sandbox in a Virtualbox on Win7(32bit)+2CPU. It is rather poor machine that the sandbox is very very slow, e.g. load average is constantly >20. I would like to stop all unnecessary daemons just to try spark-shell CLI.

YARN, Hive, Oozie and HDFS are most heavy daemons but I cannot find out if spark-shell need them or not.

1 ACCEPTED SOLUTION

avatar
Master Guru

Yes, you can stop Oozie and Hive, also HBase and Ranger if they are running. Then, depending on how much RAM do you have on your laptop, you can check and reduce Base Memory in your VM's Settings-->System. The default for Sandbox on Mac is 8G, not sure about Windows. If you are just after spark-shell you can also stop Ambari (ambari-server and ambari-agent).

View solution in original post

2 REPLIES 2

avatar
Contributor

Basically Yarn and HDFS are required if you want to run spark shell on Yarn, for others it depends on your workload.

avatar
Master Guru

Yes, you can stop Oozie and Hive, also HBase and Ranger if they are running. Then, depending on how much RAM do you have on your laptop, you can check and reduce Base Memory in your VM's Settings-->System. The default for Sandbox on Mac is 8G, not sure about Windows. If you are just after spark-shell you can also stop Ambari (ambari-server and ambari-agent).