About hedy

hedy · ‎04-15-2018

Thanks. I really appreciate your response. My advisor actually found out that this will work if we use the following command: $ pysark --master local[i] where i is a number. Using this command, multiple pyspark shells could run concurrently. But why the other solutions did not work, I have no clue!

hedy · ‎04-09-2018

It looks like things cannot run in parallel but more in a queue form. Maybe missed/misconfgured something in the installation process.

hedy · ‎04-09-2018

@saranvisa Just tried that. It's not working for different users either.

hedy · ‎04-09-2018

@saranvisa Sorry forgot to mention that... yes I did. The port is open.

hedy · ‎04-09-2018

Thank you for your help. I tried different ports, but it still doesn't work,unless I kill the running session and start another one. Can it be that I had wrong configuriation(s) during cloudera installation? Or changes needed to be made in any configuration files or somewhere else?

hedy · ‎04-07-2018

I am new to coudera. I have installed cloudera express on a Centos 7 VM, and created a cluster with 4 nodes(another 4 VMs). I ssh to the master node and run: pyspark This works but only for one session. If I open another console and run pyspark I will get the following error: WARN util.Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041. And it gets stuck there and does nothing until I close the other session running pyspark! Any idea why this is happening and how I can fix this so multiple sessions/user can run pyspark? Am I missing some configurations somewhere? Thanks in advance for your help.

Online	Offline
Last Visited	‎04-16-2018 02:06 AM

Member Since	‎04-07-2018 12:55 PM
Last Visited	‎04-16-2018 02:06 AM
Posts	6

Cloudera Community

Re: Unable to run multiple pyspark sessions

Re: Unable to run multiple pyspark sessions

Re: Unable to run multiple pyspark sessions

Re: Unable to run multiple pyspark sessions

Re: Unable to run multiple pyspark sessions

Unable to run multiple pyspark sessions