01-05-2017 11:57 AM
Using Python version 2.6.6 (r266:84292, Jul 23 2015 15:22:56)
SparkContext available as sc.
>>> from pyspark.mllib.clustering import KMeans
RuntimeError: module compiled against API version 0xa but this version
of numpy is 0x7
01-05-2017 12:22 PM
As the error implies, you probably have a different version of numpy installed in your Python environment than Pyspark expects. Consider deploying the Anaconda parcel for CDH provided by Continuum to deploy a Python env that works with Pyspark.
01-08-2017 12:57 AM
thank you for you response.
this problem has become urgent for me
can you send me the instructions how to do so?
also, my cluster version is very old 5.3.5 and spark version is 1.2
can I do this without upgrading spark or the entire cluster?
(I can't upgrade all cluster at the moment)
01-08-2017 01:07 AM
thank you again
this solution will not work for me as my spark version is too old
I first tied to install numpy 1.11 and got error using mllib library
now getting this error when trying to use kmeans library
is there any workaround to solve the compability between spark 1.2 and numpy and mllib version
without upgrading the cluster?