Support Questions
Find answers, ask questions, and share your expertise

problem about building impala

Explorer

Hi all,

 

Why the impalad from the CDH parcel can’t be replaced the one built by myself?

 

When I build impala with “-build_static_libs”, the size of impalad built is about 260M, but the one from the CDH parcel is 23M

 

When I use the “-build_shared_libs”, the size of impalad built is about 38M, but it have different *.so with the one from the CDH parcel.

 

Thank you for your time on this. I hope your response as soon as possible.

 

Best wishes.

 

Below is the information about “ldd”

    With “-build_shared_libs”:

linux-vdso.so.1 =>  (0x00007fff2a9ff000)

        libjsig.so => /usr/java/jdk1.7.0_79/jre/lib/amd64/libjsig.so (0x00007f713d918000)

        libCatalog.so => /home/test/opt/impala/be/build/debug/catalog/libCatalog.so (0x00007f713d5a3000)

        libCodeGen.so => /home/test/opt/impala/be/build/debug/codegen/libCodeGen.so (0x00007f713d2d4000)

        libCommon.so => /home/test/opt/impala/be/build/debug/common/libCommon.so (0x00007f713d094000)

        libExec.so => /home/test/opt/impala/be/build/debug/exec/libExec.so (0x00007f713c96f000)

        libExprs.so => /home/test/opt/impala/be/build/debug/exprs/libExprs.so (0x00007f713c4dc000)

        libGlobalFlags.so => /home/test/opt/impala/be/build/debug/common/libGlobalFlags.so (0x00007f713c2d9000)

        libImpalaThrift.so => /home/test/opt/impala/be/build/debug/thrift/libImpalaThrift.so (0x00007f713badf000)

        libResourceBroker.so => /home/test/opt/impala/be/build/debug/resourcebroker/libResourceBroker.so (0x00007f713b76f000)

        libRpc.so => /home/test/opt/impala/be/build/debug/rpc/libRpc.so (0x00007f713b3bd000)

        libRuntime.so => /home/test/opt/impala/be/build/debug/runtime/libRuntime.so (0x00007f713abdf000)

        libService.so => /home/test/opt/impala/be/build/debug/service/libService.so (0x00007f713a570000)

        libStatestore.so => /home/test/opt/impala/be/build/debug/statestore/libStatestore.so (0x00007f713a048000)

        libTestUtil.so => /home/test/opt/impala/be/build/debug/testutil/libTestUtil.so (0x00007f7139d97000)

        libThriftSaslTransport.so => /home/test/opt/impala/be/build/debug/transport/libThriftSaslTransport.so (0x00007f7139b46000)

        libUdf.so => /home/test/opt/impala/be/build/debug/udf/libUdf.so (0x00007f7139915000)

        libUtil.so => /home/test/opt/impala/be/build/debug/util/libUtil.so (0x00007f713942c000)

        libboost_thread.so.1.55.0 => /usr/local/lib/libboost_thread.so.1.55.0 (0x00007f7139213000)

        libboost_regex.so.1.55.0 => /usr/local/lib/libboost_regex.so.1.55.0 (0x00007f7138f26000)

        libboost_system.so.1.55.0 => /usr/local/lib/libboost_system.so.1.55.0 (0x00007f7138d23000)

        libboost_filesystem.so.1.55.0 => /usr/local/lib/libboost_filesystem.so.1.55.0 (0x00007f7138b0c000)

        libdl.so.2 => /lib64/libdl.so.2 (0x00007f7138901000)

        libjvm.so => /usr/java/jdk1.7.0_79/jre/lib/amd64/server/libjvm.so (0x00007f7137a87000)

        libhdfs.so.0.0.0 => /home/test/opt/impala/thirdparty/hadoop-2.6.0-cdh5.4.1/lib/native/libhdfs.so.0.0.0 (0x00007f7137877000)

        libgutil.so => /home/test/opt/impala/thirdparty/gutil/libgutil.so (0x00007f71375c9000)

        libz.so.1 => /lib64/libz.so.1 (0x00007f71373b3000)

        libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f71371a1000)

        librt.so.1 => /lib64/librt.so.1 (0x00007f7136f99000)

        libboost_date_time.so.1.55.0 => /usr/local/lib/libboost_date_time.so.1.55.0 (0x00007f7136d88000)

        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7136b6a000)

        libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007f71368fe000)

        libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007f713651b000)

        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f7136214000)

        libm.so.6 => /lib64/libm.so.6 (0x00007f7135f90000)

        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f7135d7a000)

        libc.so.6 => /lib64/libc.so.6 (0x00007f71359e5000)

        /lib64/ld-linux-x86-64.so.2 (0x00007f713db1d000)

        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f71357a0000)

        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f71354b9000)

        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f71352b5000)

        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f7135088000)

        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f7134e7d000)

        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f7134c79000)

        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f7134a5f000)

        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f7134840000)

 

 

With “-build_static_libs”:

        linux-vdso.so.1 =>  (0x00007fffaf1b8000)

        /usr/java/jdk1.7.0_79//jre/lib/amd64/server/libjsig.so (0x00007f82dec0e000)

        libboost_thread.so.1.55.0 => /usr/local/lib/libboost_thread.so.1.55.0 (0x00007f82de9f6000)

        libboost_regex.so.1.55.0 => /usr/local/lib/libboost_regex.so.1.55.0 (0x00007f82de709000)

        libboost_system.so.1.55.0 => /usr/local/lib/libboost_system.so.1.55.0 (0x00007f82de506000)

        libboost_filesystem.so.1.55.0 => /usr/local/lib/libboost_filesystem.so.1.55.0 (0x00007f82de2ef000)

        libboost_date_time.so.1.55.0 => /usr/local/lib/libboost_date_time.so.1.55.0 (0x00007f82de0de000)

        libdl.so.2 => /lib64/libdl.so.2 (0x00007f82dded3000)

        libz.so.1 => /lib64/libz.so.1 (0x00007f82ddcbc000)

        libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f82ddaab000)

        libjvm.so => /usr/java/jdk1.7.0_79/jre/lib/amd64/server/libjvm.so (0x00007f82dcc31000)

        librt.so.1 => /lib64/librt.so.1 (0x00007f82dca28000)

        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f82dc80b000)

        libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007f82dc59f000)

        libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007f82dc1bb000)

        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f82dbeb5000)

        libm.so.6 => /lib64/libm.so.6 (0x00007f82dbc31000)

        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f82dba1a000)

        libc.so.6 => /lib64/libc.so.6 (0x00007f82db686000)

        /lib64/ld-linux-x86-64.so.2 (0x00007f82dee13000)

        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f82db441000)

        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f82db15a000)

        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f82daf56000)

        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f82dad29000)

        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f82dab1e000)

        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f82da91a000)

        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f82da700000)

        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f82da4e1000)

 

from the CDH parcels

        linux-vdso.so.1 =>  (0x00007fff97b9d000)

        libjsig.so => not found

        libdl.so.2 => /lib64/libdl.so.2 (0x00000032f6800000)

        libz.so.1 => /lib64/libz.so.1 (0x00000032f7400000)

        libbz2.so.1 => /lib64/libbz2.so.1 (0x00000032ffc00000)

        libjvm.so => not found

        librt.so.1 => /lib64/librt.so.1 (0x00000032f7800000)

        libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032f7000000)

        libssl.so.10 => /usr/lib64/libssl.so.10 (0x00000032fe800000)

        libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00000032fa400000)

        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000032f8000000)

        libm.so.6 => /lib64/libm.so.6 (0x00000032f7c00000)

        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000032f8c00000)

        libc.so.6 => /lib64/libc.so.6 (0x00000032f6c00000)

        /lib64/ld-linux-x86-64.so.2 (0x00000032f6400000)

        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00000032fd800000)

        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00000032fc800000)

        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00000032fa800000)

        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00000032fd000000)

        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00000032fd400000)

        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00000032fcc00000)

        libresolv.so.2 => /lib64/libresolv.so.2 (0x00000032f8800000)

        libselinux.so.1 => /lib64/libselinux.so.1 (0x00000032f8400000)

1 ACCEPTED SOLUTION

Accepted Solutions

 Hi,

  There are many possible variables, including the exact version of impala, the operating system it was built on, the build flags and environment variables, and what version/build of dependencies you're using.

 

I think the specific thing you're probably seeing with file sizes is that in the CDH distribution the debug symbols are stripped from the binaries and included in separate impalad.debug files.


Are you running into some error when trying to run your custom build of Impala? It probably makes more sense to debug that problem rather than trying to exactly reproduce Cloudera's build.

View solution in original post

2 REPLIES 2

Explorer
Are there any wrong about the C Flags?
Compiler Flags: -Wall -Wno-sign-compare -Wno-unknown-pragmas -pthread -fno-strict-aliasing -Wno-deprecated -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG -g -Wno-unused-local-typedefs -O3 -DNDEBUG -gdwarf-2 -fverbose-asm -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS

 Hi,

  There are many possible variables, including the exact version of impala, the operating system it was built on, the build flags and environment variables, and what version/build of dependencies you're using.

 

I think the specific thing you're probably seeing with file sizes is that in the CDH distribution the debug symbols are stripped from the binaries and included in separate impalad.debug files.


Are you running into some error when trying to run your custom build of Impala? It probably makes more sense to debug that problem rather than trying to exactly reproduce Cloudera's build.

View solution in original post