spark shell - lack of memory

[#1]

I tried for the first time the command "spark-shell" on git bash in on order to learn how to manage data with spark and hive.

I have faced a problem a memory as you can see on the picture.spark-shell.png

I am on sandbox docker hdp 2.5 trying to learn with the "hello world" case.

I have allocated 5GO of ram for my sandbox and my PC have 8GO of RAM in total.

Until now , I have not faced a memory problem

Is it compulsory to add RAM or is there a simple solution for this problem.

[#2]

@Oriane

Try stopping the services which are not in use. This helps in freeing up the memory so that you can use spark.

[#3]

Thanks @Sandeep. You are rigth for a quick solution.

[#4 · Accepted solution]

@Oriane

Unfortunately the Sandbox has a minimum requirement of 8GB of RAM. It tends to run better with 10-12GB. Given that your computer only has 8GB of RAM total, you are going to have issues trying to run the Sandbox.

As @Sandeep Nemuri mentioned, you can try to turn off HDP components that you don't need to use. This will save some memory, but I believe you'll still have issues.

[#5]

Thanks @michael.

You are rigth for a long term solution.