Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Why foreach println on spark RDD is not working in Cluster mode.?

Highlighted

Why foreach println on spark RDD is not working in Cluster mode.?

Explorer

Hi,

 

May be this is expected, but why spark foreach println is not printing any thing in cluster mode.?

 

Below is the small snippet.

 

sc.parallelize(Seq(1,2,4))

 

when i tried sc.foreach(println), it does nothing. however, count is showing proper count. What have i done mistake..? The same works well in local mode. I would like to know the reason and workaround.

 

Thank You,

Regards, 

Srini.

1 REPLY 1

Re: Why foreach println on spark RDD is not working in Cluster mode.?

Master Collaborator
It depends on exactly what you mean by cluster mode, but you're
actually asking it to print on the executors. In local[*] mode it's
all in one local JVM but actually you are requesting the printing to
occur on the remote machines.
Don't have an account?
Coming from Hortonworks? Activate your account here