Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Impala UDF unable to load the .so file from HDFS

Solved Go to solution

Impala UDF unable to load the .so file from HDFS

Explorer

Hello Team,

 

We are planning to Impala UDFs and UDAs. To try things out we started by exploring the examples given in the cloudera github repo.

 

After building the .so file using the make utility when I try to create the function using the following

 

create function has_vowels (string) returns boolean location '/user/hive/udfs/libudfsample.so' symbol='HasVowels';

 

    We are getting the following error

 

ERROR: AnalysisException: Could not load binary: /<hdfs_path>/udfs/libudfsample.so
Unable to load /var/lib/impala/udfs/libudfsample.9909.1.so
dlerror: /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/impala/lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /var/lib/impala/udfs/libudfsample.9909.1.so)

 

 

Following are the error logs

 

org.apache.impala.common.AnalysisException: Could not load binary: /user/parth.khatwani/udfs/libudfsample.so
Unable to load /var/lib/impala/udfs/libudfsample.9909.1.so
dlerror: /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/impala/lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /var/lib/impala/udfs/libudfsample.9909.1.so)

	at org.apache.impala.catalog.Function.lookupSymbol(Function.java:442)
	at org.apache.impala.analysis.CreateUdfStmt.analyze(CreateUdfStmt.java:92)
	at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:451)
	at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:421)
	at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1285)
	at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1252)
	at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1222)
	at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:167)
I1009 02:42:52.638576 15136 status.cc:124] 3c48433373076c77:30eff6ff00000000] AnalysisException: Could not load binary: /user/parth.khatwani/udfs/libudfsample.so
Unable to load /var/lib/impala/udfs/libudfsample.9909.1.so

 

I am unable to figure out what's wrong here.

 

 

Regards

Parth

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Impala UDF unable to load the .so file from HDFS

Explorer

@Tim Armstrong  I am using gcc version 5.4.0 and  OS is ubuntu 16.04 xenial. Will it work If I compile it with 4.9.2?

View solution in original post

4 REPLIES 4
Highlighted

Re: Impala UDF unable to load the .so file from HDFS

What OS and compiler version are you using to build the UDF? This looks like it is probably a consequence of it being built with a newer gcc version than the one use to build Impala (gcc 4.9.2)

Highlighted

Re: Impala UDF unable to load the .so file from HDFS

Explorer

@Tim Armstrong  I am using gcc version 5.4.0 and  OS is ubuntu 16.04 xenial. Will it work If I compile it with 4.9.2?

View solution in original post

Highlighted

Re: Impala UDF unable to load the .so file from HDFS

@parthkyeah I'd expect so. Sometimes this C++ inter-version compatibility is a bear.

 

Highlighted

Re: Impala UDF unable to load the .so file from HDFS

Explorer

@Tim Armstrong  it worked like charm after changing the gcc version. Thanks

Don't have an account?
Coming from Hortonworks? Activate your account here