pyspark not working in my jupyter notebook

bsaad — Tue, 11 May 2021 12:00:44 GMT

Hi All,

I am doing this course on Cloudera which requires installation of anaconda, spark. After many attempts, i have successfully installed all packages but i still get this error in my notebook.

import pyspark

---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-7-c15ae3402d12> in <module>()
----> 1 import pyspark

/usr/lib/spark/python/pyspark/__init__.py in <module>()
39
40 from pyspark.conf import SparkConf
---> 41 from pyspark.context import SparkContext
42 from pyspark.rdd import RDD
43 from pyspark.files import SparkFiles

/usr/lib/spark/python/pyspark/context.py in <module>()
22 from tempfile import NamedTemporaryFile
23
---> 24 from pyspark import accumulators
25 from pyspark.accumulators import Accumulator
26 from pyspark.broadcast import Broadcast

/usr/lib/spark/python/pyspark/accumulators.py in <module>()
89 import select
90 import struct
---> 91 import SocketServer
92 import threading
93 from pyspark.cloudpickle import CloudPickler

ImportError: No module named 'SocketServer'

Can someone please help me ? Thank you

Re: pyspark not working in my jupyter notebook

bsaad — Wed, 12 May 2021 13:14:40 GMT

Hi all,

pyspark is still not recognised in my jupyter notebook for instance when I do:

import pyspark or anything related i get an error.

I have also installed many versions of python. I try open a new notebook with different python version but still facing trouble.

Here copy and paste for my .bashrc file. Please can you tell me where my error is :

# .bashrc # User specific aliases and functions alias rm='rm -i' alias cp='cp -i' alias mv='mv -i' # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi export SPARK_CLASSPATH=/usr/share/java/postgresql-jdbc-8.4.704.jar export PYSPARK_DRIVER_PYTHON_OPTS="notebook" export PYSPARK_DRIVER_PYTHON=jupyter export SPARK_CLASSPATH=/usr/share/java/postgresql-jdbc-8.4.704.jar export PYSPARK_DRIVER_PYTHON_OPTS="notebook" export PYSPARK_DRIVER_PYTHON=jupyter export PATH=$PATH:/home/cloudera/anaconda3/bin # added by Anaconda3 4.0.0 installer export PATH="/home/cloudera/anaconda3/bin:$PATH" export SPARK_CLASSPATH=/usr/share/java/postgresql-jdbc-8.4.704.jar export PYSPARK_DRIVER_PYTHON_OPTS="notebook" export PYSPARK_DRIVER_PYTHON=jupyter export SPARK_CLASSPATH=/usr/share/java/postgresql-jdbc-8.4.704.jar export PYSPARK_DRIVER_PYTHON_OPTS="notebook" export PYSPARK_DRIVER_PYTHON=jupyter

Thank you

question Re: pyspark not working in my jupyter notebook in Support Questions

pyspark not working in my jupyter notebook

Re: pyspark not working in my jupyter notebook