Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)

If you have erasure coded some directory and perform some operations on the directory you might have observed WARN messages like below

WARN erasurecode.ErasureCodeNative: Loading ISA-L failed: Failed to load libisal.so.2 (libisal.so.2: cannot open shared object file: No such file or directory)
WARN erasurecode.ErasureCodeNative: ISA-L support is not available in your platform... using builtin-java codec where applicable

This WARN messages are due to ISA Library not being present on the node.

Below are the steps to enable the library

1) Clone the isa-l github repository.

# git clone https://github.com/01org/isa-l.git

2) Go to the cloned directory

# cd isa-l

3) Install yasm if you do not have it already

# yum install -y yasm  ---> centOS
# apt-get install yasm  ----> ubuntu

4) Build the library

# make -f Makefile.unx

5) Copy the library files to lib directory

# cp bin/libisal.so bin/libisal.so.2 /lib64

6) Verify that isa-l library is enabled properly

# hadoop checknative
Expected output
18/10/12 10:20:03 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native
18/10/12 10:20:03 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop:  true /usr/hdp/3.0.0.0-1634/hadoop/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
zstd  :  false
snappy:  true /usr/hdp/3.0.0.0-1634/hadoop/lib/native/libsnappy.so.1
lz4:     true revision:10301
bzip2:   true /lib64/libbz2.so.1
openssl: true /lib64/libcrypto.so
ISA-L:   true /lib64/libisal.so.2     ------------->  Shows that ISA-L is loaded.

If step 6 uses /usr/lib64 directory instead of /lib64, you need copy the .so files in Step 5 to /usr/lib64 directory. Perform the steps on all datanode and namenode hosts or copy the .so files from the above node to /lib64 directories of all other nodes.

.

Hope this helps :)

804 Views
Comments
Cloudera Employee

Tested in my cluster as well and was able to add ISA-L libs and remove the WARN messages.

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎10-16-2018 03:27 AM
Updated by:
 
Contributors
Top Kudoed Authors