Support Questions

Find answers, ask questions, and share your expertise

Building Spark From Source

Explorer

While running spark ml, i am getting those warnings:

WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS

Some information on the web states (https://stackoverflow.com/questions/43286771/spark-with-openblas-on-emr/43398522 and this http://www.spark.tc/blas-libraries-in-mllib/ ) indicates to build spark from source.

How can I easily build from source in hdp ? Is it possible to build and then replace the hdp spark folder, without breaking stuff ? Thanks.

2 REPLIES 2

@nicolas paris Could you please confirm if you have libgfortran native library installed in your system?

Explorer

yes, I do : "Package libgfortran-4.8.5-16.el7_4.1.x86_64 already installed and latest version"