hi,adar:     if both the WAL segments and the CFiles are copied duing a tablet copy,then the follower tablet will  alse flushing wal data to disk when growing up to 8M,in my opinion  there has no difference between master tablet and follower tablet during the reading and writing,is that right? ... View more

Hello   Did you have to update the version or how did you solve the problem?   Thank you very much ... View more

I am able to sqoop the data from Oracle to HDFS and then do a create table as select * from on Impala to write into Kudu.I am abe to manually run the queries here but What is the best way to  automate this when i move the code to production.   ... View more

I'm terribly sorry for the necro-bump. The issue I'm experiencing seems to be the same one the OP was facing. We're running CDH 6.3, and I encountered a problem when trying to install the Kudu Python client. I created the symlinks as recommended by mpercy, but I'm still unable to install the kudu-python client. Our cluster does not have direct access to the internet, so when possible we use an offline install. The official documentation says that the Kudu C++ client libraries and headers are needed for the Kudu Python client. On Oracle Linux 7 trying to install devtoolset-3-toolchain ends in a failure, as a number of dependencies are missing:   Error: Package: devtoolset-3-gdb-7.8.2-38.el6.x86_64 (rhscl-devtoolset-3-epel-6-x86_64) Requires: libpython2.6.so.1.0()(64bit) Error: Package: devtoolset-3-gcc-gfortran-4.9.2-6.el6.x86_64 (rhscl-devtoolset-3-epel-6-x86_64) Requires: libmpfr.so.1()(64bit) Error: Package: devtoolset-3-gcc-c++-4.9.2-6.el6.x86_64 (rhscl-devtoolset-3-epel-6-x86_64) Requires: libgmp.so.3()(64bit) Error: Package: devtoolset-3-gcc-4.9.2-6.el6.x86_64 (rhscl-devtoolset-3-epel-6-x86_64) Requires: libgmp.so.3()(64bit) Error: Package: devtoolset-3-gcc-4.9.2-6.el6.x86_64 (rhscl-devtoolset-3-epel-6-x86_64) Requires: libmpfr.so.1()(64bit) Error: Package: devtoolset-3-gcc-gfortran-4.9.2-6.el6.x86_64 (rhscl-devtoolset-3-epel-6-x86_64) Requires: libgmp.so.3()(64bit) Error: Package: devtoolset-3-gcc-c++-4.9.2-6.el6.x86_64 (rhscl-devtoolset-3-epel-6-x86_64) Requires: libmpfr.so.1()(64bit) Disregarding that, running pip install --no-index --find-links file:///data0/home/jkovacs/kudu-python-1.10.0.tar.gz  results in the following errors: ERROR: Command errored out with exit status 1: command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-rK2Qbk/kudu-python/setup.py'"'"'; __file__='"'"'/tmp/pip-install-rK2Qbk/kudu-python/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-rK2Qbk/kudu-python/pip-egg-info cwd: /tmp/pip-install-rK2Qbk/kudu-python/ Complete output (43 lines): Building from system prefix /usr/local /usr/lib64/python2.7/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-rK2Qbk/kudu-python/kudu/client.pxd tree = Parsing.p_module(s, pxd, full_module_name) /usr/lib64/python2.7/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-rK2Qbk/kudu-python/kudu/schema.pxd tree = Parsing.p_module(s, pxd, full_module_name) /usr/lib64/python2.7/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-rK2Qbk/kudu-python/kudu/errors.pxd tree = Parsing.p_module(s, pxd, full_module_name) Compiling kudu/client.pyx because it depends on kudu/config.pxi. Compiling kudu/errors.pyx because it depends on /usr/lib64/python2.7/site-packages/Cython/Includes/libcpp/string.pxd. Compiling kudu/schema.pyx because it depends on kudu/config.pxi. [1/3] Cythonizing kudu/client.pyx [2/3] Cythonizing kudu/schema.pyx [3/3] Cythonizing kudu/errors.pyx WARNING: The wheel package is not available. DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at <a href="<a href="https://pip.pypa.io/en/latest/development/release-process/#python-2-support" target="_blank">https://pip.pypa.io/en/latest/development/release-process/#python-2-support</a>" target="_blank"><a href="https://pip.pypa.io/en/latest/development/release-process/#python-2-support</a" target="_blank">https://pip.pypa.io/en/latest/development/release-process/#python-2-support</a</a>> WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe8bbca3910>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/pytest-runner/ WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe8bbca3110>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/pytest-runner/ WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe8bbca3090>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/pytest-runner/ WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe8bbca9e50>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/pytest-runner/ WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe8bbca9e90>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/pytest-runner/ ERROR: Could not find a version that satisfies the requirement pytest-runner (from versions: none) ERROR: No matching distribution found for pytest-runner Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-install-rK2Qbk/kudu-python/setup.py", line 216, in <module> test_suite="kudu.tests" File "/usr/lib/python2.7/site-packages/setuptools/__init__.py", line 144, in setup _install_setup_requires(attrs) File "/usr/lib/python2.7/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires dist.fetch_build_eggs(dist.setup_requires) File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 721, in fetch_build_eggs replace_conflicting=True, File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve replace_conflicting=replace_conflicting File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match return self.obtain(req, installer) File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain return installer(requirement) File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 777, in fetch_build_egg return fetch_build_egg(self, req) File "/usr/lib/python2.7/site-packages/setuptools/installer.py", line 121, in fetch_build_egg raise DistutilsError(str(e)) distutils.errors.DistutilsError: Command '['/usr/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmp8eqkO2', '--quiet', 'pytest-runner']' returned non-zero exit status 1 ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. I'd like to ask for some guidance, what to try next?     Thank you!   ... View more

Let's try to rule out various types of problems.   1. Are you able to read/write to Kerberos-enabled HDFS with PySpark? Is Kudu the only Kerberos-enabled service that is not working from within PySpark?   2. Have you checked to ensure that the Spark driver is running on the host and shell you kinited from instead of being started in a YARN container? If it's running in YARN you have to give YARN access to the keytab to run as.   3. Have you tried connecting to Kudu with the regular Spark shell? Does it work? For examples see https://kudu.apache.org/docs/developing.html#_kudu_integration_with_spark ... View more

Kudu runs as a separate service that Impala talks to (like HDFS runs as a separate service from Impala) so you have to have Kudu running somewhere for it to work. However you don't have to run Kudu on the same servers that you run Impala on -- remote reads are supported over the network. ... View more

I tried the alter command below in impala-shell 2.12.0 and kudu 1.7.0. However, I'm getting an error. My table is an external table in impala.   The error message is strange. Of course the new table doesn't exists, I want to create it with the command...   ALTER TABLE res_dhcp_int SET TBLPROPERTIES('kudu.table_name'='res_dhcp_int'); Query: ALTER TABLE res_dhcp_int SET TBLPROPERTIES('kudu.table_name'='res_dhcp_int') ERROR: TableLoadingException: Error loading metadata for Kudu table res_dhcp_int CAUSED BY: ImpalaRuntimeException: Error opening Kudu table 'res_dhcp_int', Kudu error: The table does not exist: table_name: "res_dhcp_int"   is this a bug?    EDIT: i just read IMPALA-5654, seems that with impala 2.12.0 this alter command doesn't work anymore! I need an alternative for that 😞 ... View more

thanks for these informations, a little question : where must I define the custom mapping ?   Best regards   Christophe ... View more

@Andreyeff Another thing you can try doing is increasing the raft heartbeat interval from 500ms to 1500ms or even 3000ms, see https://kudu.apache.org/docs/configuration_reference.html#kudu-tserver_raft_heartbeat_interval_ms   This will affect your recovery time by a few seconds if a leader fails since by default, elections don't happen for 3 missed heartbeat periods (controlled by https://kudu.apache.org/docs/configuration_reference.html#kudu-tserver_leader_failure_max_missed_heartbeat_periods ) ... View more

thank you again for the solution.I found a java code which loads data into kudu table from csv. is it efficient.?  ... View more