Reply
Highlighted
New Contributor
Posts: 3
Registered: ‎05-11-2017

How to Set PathHashSeed

I just recently setup my cluster and when I attempt to run a spark job using python I am getting errors about PYTHONHASHSEED needing to be set.  Using the Spark config UI in CM I added the following two lines to my spark-env.sh.  I also added them to my ~/.bashrc on each node.

 

 

export PYSPARK_PYTHON=python3

export PYTHONHASHSEED=8675309

 

Unfortunately, I am still getting errors.  Any advice?

Announcements