Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Building Spark From Source

Building Spark From Source

New Contributor

While running spark ml, i am getting those warnings:

WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS

Some information on the web states (https://stackoverflow.com/questions/43286771/spark-with-openblas-on-emr/43398522 and this http://www.spark.tc/blas-libraries-in-mllib/ ) indicates to build spark from source.

How can I easily build from source in hdp ? Is it possible to build and then replace the hdp spark folder, without breaking stuff ? Thanks.

2 REPLIES 2

Re: Building Spark From Source

@nicolas paris Could you please confirm if you have libgfortran native library installed in your system?

Highlighted

Re: Building Spark From Source

New Contributor

yes, I do : "Package libgfortran-4.8.5-16.el7_4.1.x86_64 already installed and latest version"

Don't have an account?
Coming from Hortonworks? Activate your account here