Support Questions
Find answers, ask questions, and share your expertise

Install spark over the quickstart vm

Explorer

Hi,

 

I would like to Install spark over the quickstart vm. any explaination how?

 

Thanks in advance,

Tal

11 REPLIES 11

Re: Install spark over the quickstart vm

Master Collaborator

talgo,

 

You may install a Spark parcel using the CM instance inside the current VM:

 

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Instal...

 

Our next version of the VM, for Cloudera Enterprise 5, will contain Spark natively.

Re: Install spark over the quickstart vm

Explorer

thanks! do i need to upgrade to cdh 4.6 first?

Re: Install spark over the quickstart vm

Guru

No, since there isn't something to upgrade 😉

You're creating a new vm from scratch. You do not need to take the latest available quickstart vm as a starting point.

Re: Install spark over the quickstart vm

Explorer

hi ge-ko,

sorry, but i really don't understand the instruction of 'build your own CDH5' .  it seems it has assumptions the reader has some experience with centos. i tried to ask google where i didn't have the answer in the instruction but i feel hopeless..

 

In addition, I tried to install spark parcel over the quick vm - but when i asked to update the envoirment params in the spark_env.sh - > i set the HADOOP_HOME as i asked to but still had problems with running the master (remember - i use only 1 node).

 

hopeless... Smiley Indifferent

Re: Install spark over the quickstart vm

Guru

Hi,

 

seems like you're thinking too complicated 😉

You are not forced to use CentOS. If you are familiar with Ubuntu, just create a new vm with plain Ubuntu installation. Afterwards install ClouderaManager5 (like described here) and choose installation Path A.

To put it in a nutshell:

1.) create a new vm with base OS installed

2.) ensure prerequisites are fulfilled before installing ClouderaManager5 (see link above)

3.) install CDH5 including Spark by using ClouderaManager on your singlenode "cluster" (==the recently created vm)

 

done.

 

Or, btw., since CDH5 GA is available since yesterday possibly there will be an upgraded quickstart vm released soon ?!?

 

hth 

Re: Install spark over the quickstart vm

Explorer

 ubuntu -> trying to install the cloudera manager and got this message suddenly.

 

ubuntu.PNG

 

i checked the log :

ubuntu2.PNG

 

 

then i tried to install postgresql but still i had the same issue.. 😞

Re: Install spark over the quickstart vm

Guru

Hi talgo,

 

your log/error messages are invisible =>

 

missing-pic.png

 

 

Please try it again, or just paste the output from command line

Re: Install spark over the quickstart vm

Explorer

ok i step forward in the installation.. now my problem is what to set in the host ip.. i tried 127.0.0.1 but it can't add it cause it can't connect via ssh...

ubuntu3.PNG

Re: Install spark over the quickstart vm

Explorer
ok i resolved this issue... will update later