Reply
Explorer
Posts: 7
Registered: ‎05-08-2016

Getting exception when running MLlib on python 2.6.6 using numpy 1.7

Using Python version 2.6.6 (r266:84292, Jul 23 2015 15:22:56)
SparkContext available as sc.
>>> from pyspark.mllib.clustering import KMeans
RuntimeError: module compiled against API version 0xa but this version
of numpy is 0x7

Cloudera Employee
Posts: 481
Registered: ‎08-11-2014

Re: Getting exception when running MLlib on python 2.6.6 using numpy 1.7

As the error implies, you probably have a different version of numpy installed in your Python environment than Pyspark expects. Consider deploying the Anaconda parcel for CDH provided by Continuum to deploy a Python env that works with Pyspark.

Explorer
Posts: 7
Registered: ‎05-08-2016

Re: Getting exception when running MLlib on python 2.6.6 using numpy 1.7

Hi

 

thank you for you response.

this problem has become urgent for me

can you send me the instructions how to do so?

also, my cluster version is very old 5.3.5 and spark version is 1.2

can I do this without upgrading spark or the entire cluster?

(I can't upgrade all cluster at the moment)

Explorer
Posts: 7
Registered: ‎05-08-2016

Re: Getting exception when running MLlib on python 2.6.6 using numpy 1.7

hi

 

thank you again

this solution will not work for me as my spark version is too old

I first tied to install numpy 1.11 and got error using mllib library

now getting this error when trying to use kmeans library

is there any workaround to solve the compability between spark 1.2 and numpy and mllib version

without upgrading the cluster?