Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Impala UDF unable to load the .so file from HDFS

avatar
Contributor

Hello Team,

 

We are planning to Impala UDFs and UDAs. To try things out we started by exploring the examples given in the cloudera github repo.

 

After building the .so file using the make utility when I try to create the function using the following

 

create function has_vowels (string) returns boolean location '/user/hive/udfs/libudfsample.so' symbol='HasVowels';

 

    We are getting the following error

 

ERROR: AnalysisException: Could not load binary: /<hdfs_path>/udfs/libudfsample.so
Unable to load /var/lib/impala/udfs/libudfsample.9909.1.so
dlerror: /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/impala/lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /var/lib/impala/udfs/libudfsample.9909.1.so)

 

 

Following are the error logs

 

org.apache.impala.common.AnalysisException: Could not load binary: /user/parth.khatwani/udfs/libudfsample.so
Unable to load /var/lib/impala/udfs/libudfsample.9909.1.so
dlerror: /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/impala/lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /var/lib/impala/udfs/libudfsample.9909.1.so)

	at org.apache.impala.catalog.Function.lookupSymbol(Function.java:442)
	at org.apache.impala.analysis.CreateUdfStmt.analyze(CreateUdfStmt.java:92)
	at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:451)
	at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:421)
	at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1285)
	at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1252)
	at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1222)
	at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:167)
I1009 02:42:52.638576 15136 status.cc:124] 3c48433373076c77:30eff6ff00000000] AnalysisException: Could not load binary: /user/parth.khatwani/udfs/libudfsample.so
Unable to load /var/lib/impala/udfs/libudfsample.9909.1.so

 

I am unable to figure out what's wrong here.

 

 

Regards

Parth

1 ACCEPTED SOLUTION

avatar
Contributor

@Tim Armstrong  I am using gcc version 5.4.0 and  OS is ubuntu 16.04 xenial. Will it work If I compile it with 4.9.2?

View solution in original post

4 REPLIES 4

avatar

What OS and compiler version are you using to build the UDF? This looks like it is probably a consequence of it being built with a newer gcc version than the one use to build Impala (gcc 4.9.2)

avatar
Contributor

@Tim Armstrong  I am using gcc version 5.4.0 and  OS is ubuntu 16.04 xenial. Will it work If I compile it with 4.9.2?

avatar

@parthkyeah I'd expect so. Sometimes this C++ inter-version compatibility is a bear.

 

avatar
Contributor

@Tim Armstrong  it worked like charm after changing the gcc version. Thanks