Created on 02-23-2016 07:32 PM - edited 09-16-2022 03:05 AM
Hi all,
Why the impalad from the CDH parcel can’t be replaced the one built by myself?
When I build impala with “-build_static_libs”, the size of impalad built is about 260M, but the one from the CDH parcel is 23M
When I use the “-build_shared_libs”, the size of impalad built is about 38M, but it have different *.so with the one from the CDH parcel.
Thank you for your time on this. I hope your response as soon as possible.
Best wishes.
Below is the information about “ldd”
With “-build_shared_libs”:
linux-vdso.so.1 => (0x00007fff2a9ff000)
libjsig.so => /usr/java/jdk1.7.0_79/jre/lib/amd64/libjsig.so (0x00007f713d918000)
libCatalog.so => /home/test/opt/impala/be/build/debug/catalog/libCatalog.so (0x00007f713d5a3000)
libCodeGen.so => /home/test/opt/impala/be/build/debug/codegen/libCodeGen.so (0x00007f713d2d4000)
libCommon.so => /home/test/opt/impala/be/build/debug/common/libCommon.so (0x00007f713d094000)
libExec.so => /home/test/opt/impala/be/build/debug/exec/libExec.so (0x00007f713c96f000)
libExprs.so => /home/test/opt/impala/be/build/debug/exprs/libExprs.so (0x00007f713c4dc000)
libGlobalFlags.so => /home/test/opt/impala/be/build/debug/common/libGlobalFlags.so (0x00007f713c2d9000)
libImpalaThrift.so => /home/test/opt/impala/be/build/debug/thrift/libImpalaThrift.so (0x00007f713badf000)
libResourceBroker.so => /home/test/opt/impala/be/build/debug/resourcebroker/libResourceBroker.so (0x00007f713b76f000)
libRpc.so => /home/test/opt/impala/be/build/debug/rpc/libRpc.so (0x00007f713b3bd000)
libRuntime.so => /home/test/opt/impala/be/build/debug/runtime/libRuntime.so (0x00007f713abdf000)
libService.so => /home/test/opt/impala/be/build/debug/service/libService.so (0x00007f713a570000)
libStatestore.so => /home/test/opt/impala/be/build/debug/statestore/libStatestore.so (0x00007f713a048000)
libTestUtil.so => /home/test/opt/impala/be/build/debug/testutil/libTestUtil.so (0x00007f7139d97000)
libThriftSaslTransport.so => /home/test/opt/impala/be/build/debug/transport/libThriftSaslTransport.so (0x00007f7139b46000)
libUdf.so => /home/test/opt/impala/be/build/debug/udf/libUdf.so (0x00007f7139915000)
libUtil.so => /home/test/opt/impala/be/build/debug/util/libUtil.so (0x00007f713942c000)
libboost_thread.so.1.55.0 => /usr/local/lib/libboost_thread.so.1.55.0 (0x00007f7139213000)
libboost_regex.so.1.55.0 => /usr/local/lib/libboost_regex.so.1.55.0 (0x00007f7138f26000)
libboost_system.so.1.55.0 => /usr/local/lib/libboost_system.so.1.55.0 (0x00007f7138d23000)
libboost_filesystem.so.1.55.0 => /usr/local/lib/libboost_filesystem.so.1.55.0 (0x00007f7138b0c000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f7138901000)
libjvm.so => /usr/java/jdk1.7.0_79/jre/lib/amd64/server/libjvm.so (0x00007f7137a87000)
libhdfs.so.0.0.0 => /home/test/opt/impala/thirdparty/hadoop-2.6.0-cdh5.4.1/lib/native/libhdfs.so.0.0.0 (0x00007f7137877000)
libgutil.so => /home/test/opt/impala/thirdparty/gutil/libgutil.so (0x00007f71375c9000)
libz.so.1 => /lib64/libz.so.1 (0x00007f71373b3000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f71371a1000)
librt.so.1 => /lib64/librt.so.1 (0x00007f7136f99000)
libboost_date_time.so.1.55.0 => /usr/local/lib/libboost_date_time.so.1.55.0 (0x00007f7136d88000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7136b6a000)
libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007f71368fe000)
libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007f713651b000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f7136214000)
libm.so.6 => /lib64/libm.so.6 (0x00007f7135f90000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f7135d7a000)
libc.so.6 => /lib64/libc.so.6 (0x00007f71359e5000)
/lib64/ld-linux-x86-64.so.2 (0x00007f713db1d000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f71357a0000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f71354b9000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f71352b5000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f7135088000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f7134e7d000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f7134c79000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f7134a5f000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f7134840000)
With “-build_static_libs”:
linux-vdso.so.1 => (0x00007fffaf1b8000)
/usr/java/jdk1.7.0_79//jre/lib/amd64/server/libjsig.so (0x00007f82dec0e000)
libboost_thread.so.1.55.0 => /usr/local/lib/libboost_thread.so.1.55.0 (0x00007f82de9f6000)
libboost_regex.so.1.55.0 => /usr/local/lib/libboost_regex.so.1.55.0 (0x00007f82de709000)
libboost_system.so.1.55.0 => /usr/local/lib/libboost_system.so.1.55.0 (0x00007f82de506000)
libboost_filesystem.so.1.55.0 => /usr/local/lib/libboost_filesystem.so.1.55.0 (0x00007f82de2ef000)
libboost_date_time.so.1.55.0 => /usr/local/lib/libboost_date_time.so.1.55.0 (0x00007f82de0de000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f82dded3000)
libz.so.1 => /lib64/libz.so.1 (0x00007f82ddcbc000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f82ddaab000)
libjvm.so => /usr/java/jdk1.7.0_79/jre/lib/amd64/server/libjvm.so (0x00007f82dcc31000)
librt.so.1 => /lib64/librt.so.1 (0x00007f82dca28000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f82dc80b000)
libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007f82dc59f000)
libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007f82dc1bb000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f82dbeb5000)
libm.so.6 => /lib64/libm.so.6 (0x00007f82dbc31000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f82dba1a000)
libc.so.6 => /lib64/libc.so.6 (0x00007f82db686000)
/lib64/ld-linux-x86-64.so.2 (0x00007f82dee13000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f82db441000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f82db15a000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f82daf56000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f82dad29000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f82dab1e000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f82da91a000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f82da700000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f82da4e1000)
from the CDH parcels
linux-vdso.so.1 => (0x00007fff97b9d000)
libjsig.so => not found
libdl.so.2 => /lib64/libdl.so.2 (0x00000032f6800000)
libz.so.1 => /lib64/libz.so.1 (0x00000032f7400000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00000032ffc00000)
libjvm.so => not found
librt.so.1 => /lib64/librt.so.1 (0x00000032f7800000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032f7000000)
libssl.so.10 => /usr/lib64/libssl.so.10 (0x00000032fe800000)
libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00000032fa400000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000032f8000000)
libm.so.6 => /lib64/libm.so.6 (0x00000032f7c00000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000032f8c00000)
libc.so.6 => /lib64/libc.so.6 (0x00000032f6c00000)
/lib64/ld-linux-x86-64.so.2 (0x00000032f6400000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00000032fd800000)
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00000032fc800000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00000032fa800000)
libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00000032fd000000)
libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00000032fd400000)
libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00000032fcc00000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00000032f8800000)
libselinux.so.1 => /lib64/libselinux.so.1 (0x00000032f8400000)
Created 02-23-2016 08:39 PM
Hi,
There are many possible variables, including the exact version of impala, the operating system it was built on, the build flags and environment variables, and what version/build of dependencies you're using.
I think the specific thing you're probably seeing with file sizes is that in the CDH distribution the debug symbols are stripped from the binaries and included in separate impalad.debug files.
Are you running into some error when trying to run your custom build of Impala? It probably makes more sense to debug that problem rather than trying to exactly reproduce Cloudera's build.
Created 02-23-2016 08:10 PM
Created 02-23-2016 08:39 PM
Hi,
There are many possible variables, including the exact version of impala, the operating system it was built on, the build flags and environment variables, and what version/build of dependencies you're using.
I think the specific thing you're probably seeing with file sizes is that in the CDH distribution the debug symbols are stripped from the binaries and included in separate impalad.debug files.
Are you running into some error when trying to run your custom build of Impala? It probably makes more sense to debug that problem rather than trying to exactly reproduce Cloudera's build.