- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Impala UDF unable to load the .so file from HDFS
- Labels:
-
Apache Impala
Created 10-08-2020 07:47 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Team,
We are planning to Impala UDFs and UDAs. To try things out we started by exploring the examples given in the cloudera github repo.
After building the .so file using the make utility when I try to create the function using the following
create function has_vowels (string) returns boolean location '/user/hive/udfs/libudfsample.so' symbol='HasVowels';
We are getting the following error
ERROR: AnalysisException: Could not load binary: /<hdfs_path>/udfs/libudfsample.so
Unable to load /var/lib/impala/udfs/libudfsample.9909.1.so
dlerror: /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/impala/lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /var/lib/impala/udfs/libudfsample.9909.1.so)
Following are the error logs
org.apache.impala.common.AnalysisException: Could not load binary: /user/parth.khatwani/udfs/libudfsample.so
Unable to load /var/lib/impala/udfs/libudfsample.9909.1.so
dlerror: /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/impala/lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /var/lib/impala/udfs/libudfsample.9909.1.so)
at org.apache.impala.catalog.Function.lookupSymbol(Function.java:442)
at org.apache.impala.analysis.CreateUdfStmt.analyze(CreateUdfStmt.java:92)
at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:451)
at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:421)
at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1285)
at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1252)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1222)
at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:167)
I1009 02:42:52.638576 15136 status.cc:124] 3c48433373076c77:30eff6ff00000000] AnalysisException: Could not load binary: /user/parth.khatwani/udfs/libudfsample.so
Unable to load /var/lib/impala/udfs/libudfsample.9909.1.so
I am unable to figure out what's wrong here.
Regards
Parth
Created 10-11-2020 11:53 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Tim Armstrong I am using gcc version 5.4.0 and OS is ubuntu 16.04 xenial. Will it work If I compile it with 4.9.2?
Created 10-09-2020 10:28 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What OS and compiler version are you using to build the UDF? This looks like it is probably a consequence of it being built with a newer gcc version than the one use to build Impala (gcc 4.9.2)
Created 10-11-2020 11:53 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Tim Armstrong I am using gcc version 5.4.0 and OS is ubuntu 16.04 xenial. Will it work If I compile it with 4.9.2?
Created 10-13-2020 03:44 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@parthkyeah I'd expect so. Sometimes this C++ inter-version compatibility is a bear.
Created 10-15-2020 04:13 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Tim Armstrong it worked like charm after changing the gcc version. Thanks