Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

some install and initial setup questions

some install and initial setup questions

Explorer

Hi

 

newbie question.

 

I wont be surprised if these have been asked before, but I am trying with CDH manager 5.4 on AWS to install Spark.

 

first up, the install sometimes hangs at "deploying client config" although it sucessfuly installs Oozie and several other services.

 

I downloaded the Spark parcel.

Set up the master and worker ( for worker  I specfied the master as one of the workers, not sure about that).

See page -  Installing Spark with Cloudera Manager

 

Now how do I know from the UI that my master and workers are up ?

When I run spark shell it gets into an infinite loop    INFO Client: Application report for application_1430414416377_0004 (state: ACCEPTED)

 

I tried spark submit, but the file spark-examples_version.jar is not there on disk, on either master or worker.

 

If someone can make notes about missing steps/errors, that would be great.

3 REPLIES 3

Re: some install and initial setup questions

Master Collaborator

Not sure why the deploy config step is hanging there. I'm not sure what you mean about an infinite loop. If your executors are accepted then you're running. The examples JAR is shipped, in the parcels directory. I don't know where you're looking for it.

Re: some install and initial setup questions

Explorer

Mr Owen

 

thanks for replying.

 

#deploy config step is hanging there ->

saw the log as directed by another blog post, from my memory the log said something like "persistence test passed" and then did nothing. I cannot remember if its was the same error every time. ( I tried the install several times).

 

#I'm not sure what you mean about an infinite loop ->

I mean spark shell keeps on printing that message, and does not give me a prompt. I had read some other blog post which suggested to raise the log level to higher than INFO.

 

#The examples JAR is shipped, in the parcels directory. I don't know where you're looking for it. --> 

find / -name spark-examples_version.jar

I copy pasted from the website where the name of the jar is wrong.

http://www.cloudera.com/content/cloudera/en/documentation/core/v5-2-x/topics/cdh_ig_running_spark_ap...

 

#If your executors are accepted then you're running. -->

can I see in the UI who is the master and who are the workers?

When specifying the worker IPs, I have included the master as a worker as well. Is that valid?

 

 

 

Re: some install and initial setup questions

Explorer

Mr Owen

 

I posted a separate message for the install problem.

 

http://community.cloudera.com/t5/Cloudera-Manager-Installation/AWS-install-hangs-at-Deploying-Client...