problem about building impala

heyi — Fri, 16 Sep 2022 10:05:22 GMT

Hi all,

Why the impalad from the CDH parcel can’t be replaced the one built by myself?

When I build impala with “-build_static_libs”, the size of impalad built is about 260M, but the one from the CDH parcel is 23M

When I use the “-build_shared_libs”, the size of impalad built is about 38M, but it have different *.so with the one from the CDH parcel.

Thank you for your time on this. I hope your response as soon as possible.

Best wishes.

Below is the information about “ldd”

With “-build_shared_libs”:

linux-vdso.so.1 => (0x00007fff2a9ff000)

libjsig.so => /usr/java/jdk1.7.0_79/jre/lib/amd64/libjsig.so (0x00007f713d918000)

libCatalog.so => /home/test/opt/impala/be/build/debug/catalog/libCatalog.so (0x00007f713d5a3000)

libCodeGen.so => /home/test/opt/impala/be/build/debug/codegen/libCodeGen.so (0x00007f713d2d4000)

libCommon.so => /home/test/opt/impala/be/build/debug/common/libCommon.so (0x00007f713d094000)

libExec.so => /home/test/opt/impala/be/build/debug/exec/libExec.so (0x00007f713c96f000)

libExprs.so => /home/test/opt/impala/be/build/debug/exprs/libExprs.so (0x00007f713c4dc000)

libGlobalFlags.so => /home/test/opt/impala/be/build/debug/common/libGlobalFlags.so (0x00007f713c2d9000)

libImpalaThrift.so => /home/test/opt/impala/be/build/debug/thrift/libImpalaThrift.so (0x00007f713badf000)

libResourceBroker.so => /home/test/opt/impala/be/build/debug/resourcebroker/libResourceBroker.so (0x00007f713b76f000)

libRpc.so => /home/test/opt/impala/be/build/debug/rpc/libRpc.so (0x00007f713b3bd000)

libRuntime.so => /home/test/opt/impala/be/build/debug/runtime/libRuntime.so (0x00007f713abdf000)

libService.so => /home/test/opt/impala/be/build/debug/service/libService.so (0x00007f713a570000)

libStatestore.so => /home/test/opt/impala/be/build/debug/statestore/libStatestore.so (0x00007f713a048000)

libTestUtil.so => /home/test/opt/impala/be/build/debug/testutil/libTestUtil.so (0x00007f7139d97000)

libThriftSaslTransport.so => /home/test/opt/impala/be/build/debug/transport/libThriftSaslTransport.so (0x00007f7139b46000)

libUdf.so => /home/test/opt/impala/be/build/debug/udf/libUdf.so (0x00007f7139915000)

libUtil.so => /home/test/opt/impala/be/build/debug/util/libUtil.so (0x00007f713942c000)

libboost_thread.so.1.55.0 => /usr/local/lib/libboost_thread.so.1.55.0 (0x00007f7139213000)

libboost_regex.so.1.55.0 => /usr/local/lib/libboost_regex.so.1.55.0 (0x00007f7138f26000)

libboost_system.so.1.55.0 => /usr/local/lib/libboost_system.so.1.55.0 (0x00007f7138d23000)

libboost_filesystem.so.1.55.0 => /usr/local/lib/libboost_filesystem.so.1.55.0 (0x00007f7138b0c000)

libdl.so.2 => /lib64/libdl.so.2 (0x00007f7138901000)

libjvm.so => /usr/java/jdk1.7.0_79/jre/lib/amd64/server/libjvm.so (0x00007f7137a87000)

libhdfs.so.0.0.0 => /home/test/opt/impala/thirdparty/hadoop-2.6.0-cdh5.4.1/lib/native/libhdfs.so.0.0.0 (0x00007f7137877000)

libgutil.so => /home/test/opt/impala/thirdparty/gutil/libgutil.so (0x00007f71375c9000)

libz.so.1 => /lib64/libz.so.1 (0x00007f71373b3000)

libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f71371a1000)

librt.so.1 => /lib64/librt.so.1 (0x00007f7136f99000)

libboost_date_time.so.1.55.0 => /usr/local/lib/libboost_date_time.so.1.55.0 (0x00007f7136d88000)

libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7136b6a000)

libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007f71368fe000)

libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007f713651b000)

libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f7136214000)

libm.so.6 => /lib64/libm.so.6 (0x00007f7135f90000)

libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f7135d7a000)

libc.so.6 => /lib64/libc.so.6 (0x00007f71359e5000)

/lib64/ld-linux-x86-64.so.2 (0x00007f713db1d000)

libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f71357a0000)

libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f71354b9000)

libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f71352b5000)

libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f7135088000)

libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f7134e7d000)

libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f7134c79000)

libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f7134a5f000)

libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f7134840000)

With “-build_static_libs”:

linux-vdso.so.1 => (0x00007fffaf1b8000)

/usr/java/jdk1.7.0_79//jre/lib/amd64/server/libjsig.so (0x00007f82dec0e000)

libboost_thread.so.1.55.0 => /usr/local/lib/libboost_thread.so.1.55.0 (0x00007f82de9f6000)

libboost_regex.so.1.55.0 => /usr/local/lib/libboost_regex.so.1.55.0 (0x00007f82de709000)

libboost_system.so.1.55.0 => /usr/local/lib/libboost_system.so.1.55.0 (0x00007f82de506000)

libboost_filesystem.so.1.55.0 => /usr/local/lib/libboost_filesystem.so.1.55.0 (0x00007f82de2ef000)

libboost_date_time.so.1.55.0 => /usr/local/lib/libboost_date_time.so.1.55.0 (0x00007f82de0de000)

libdl.so.2 => /lib64/libdl.so.2 (0x00007f82dded3000)

libz.so.1 => /lib64/libz.so.1 (0x00007f82ddcbc000)

libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f82ddaab000)

libjvm.so => /usr/java/jdk1.7.0_79/jre/lib/amd64/server/libjvm.so (0x00007f82dcc31000)

librt.so.1 => /lib64/librt.so.1 (0x00007f82dca28000)

libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f82dc80b000)

libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007f82dc59f000)

libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007f82dc1bb000)

libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f82dbeb5000)

libm.so.6 => /lib64/libm.so.6 (0x00007f82dbc31000)

libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f82dba1a000)

libc.so.6 => /lib64/libc.so.6 (0x00007f82db686000)

/lib64/ld-linux-x86-64.so.2 (0x00007f82dee13000)

libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f82db441000)

libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f82db15a000)

libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f82daf56000)

libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f82dad29000)

libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f82dab1e000)

libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f82da91a000)

libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f82da700000)

libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f82da4e1000)

from the CDH parcels

linux-vdso.so.1 => (0x00007fff97b9d000)

libjsig.so => not found

libdl.so.2 => /lib64/libdl.so.2 (0x00000032f6800000)

libz.so.1 => /lib64/libz.so.1 (0x00000032f7400000)

libbz2.so.1 => /lib64/libbz2.so.1 (0x00000032ffc00000)

libjvm.so => not found

librt.so.1 => /lib64/librt.so.1 (0x00000032f7800000)

libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032f7000000)

libssl.so.10 => /usr/lib64/libssl.so.10 (0x00000032fe800000)

libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00000032fa400000)

libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000032f8000000)

libm.so.6 => /lib64/libm.so.6 (0x00000032f7c00000)

libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000032f8c00000)

libc.so.6 => /lib64/libc.so.6 (0x00000032f6c00000)

/lib64/ld-linux-x86-64.so.2 (0x00000032f6400000)

libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00000032fd800000)

libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00000032fc800000)

libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00000032fa800000)

libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00000032fd000000)

libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00000032fd400000)

libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00000032fcc00000)

libresolv.so.2 => /lib64/libresolv.so.2 (0x00000032f8800000)

libselinux.so.1 => /lib64/libselinux.so.1 (0x00000032f8400000)

Re: problem about building impala

heyi — Wed, 24 Feb 2016 04:10:11 GMT

Are there any wrong about the C Flags?
Compiler Flags: -Wall -Wno-sign-compare -Wno-unknown-pragmas -pthread -fno-strict-aliasing -Wno-deprecated -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG -g -Wno-unused-local-typedefs -O3 -DNDEBUG -gdwarf-2 -fverbose-asm -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS

Re: problem about building impala

Tim Armstrong — Wed, 24 Feb 2016 04:39:56 GMT

Hi,

There are many possible variables, including the exact version of impala, the operating system it was built on, the build flags and environment variables, and what version/build of dependencies you're using.

I think the specific thing you're probably seeing with file sizes is that in the CDH distribution the debug symbols are stripped from the binaries and included in separate impalad.debug files.

Are you running into some error when trying to run your custom build of Impala? It probably makes more sense to debug that problem rather than trying to exactly reproduce Cloudera's build.

question Re: problem about building impala in Archives of Support Questions (Read Only)

problem about building impala

Re: problem about building impala

Re: problem about building impala