Re: When I add a new rack some Impala queries beca...

Enri · ‎01-10-2020

As it says in the documentation: Because this option results in increased resource utilization on a single host, it could cause problems due to contention with other Impala statements or high resource usage. Symptoms could include queries running slowly, exceeding the memory limit, or appearing to hang. Use it only in a single-user development/test environment; do not use it in a production environment or in a cluster with a high-concurrency or high-volume or performance-critical workload.

anandnalya · ‎07-25-2019

Got it working on my cluster. Thanks Lars and Eric. Cheers, Anand

RajeshGhosh · ‎05-06-2019

Worked for me too. Thank you.

stigahuang · ‎03-18-2019

I encountered this kind of issues before. It helps if you can paste your JDBC URL string. Do you run in a secured cluster? If not, you could try using AuthMech=3 and UseSasl=0 together.

alpertankut · ‎08-28-2018

there is a possibility that your daemon process is hung find it and kill it $> ps -eaf|grep impala impala 4399 1 0 Aug17 ? 00:00:00 python2.7 /usr/lib64/cmf/agent/build/env/bin/cmf-redactor /usr/lib64/cmf/service/impala/impala.sh impalad impalad_flags false clouder+ 8426 5709 0 16:34 pts/0 00:00:00 grep --color=auto impala impala 12322 1 0 Aug17 ? 00:00:00 /opt/cloudera/parcels/CDH-5.10.1-1.cdh5.10.1.p0.10/lib/impala/sbin-retail/impalad --flagfile=/run/cloudera-scm-agent/process/3639-impala-IMPALAD/impala-conf/impalad_flags $>sudo kill -9 12322 $>sudo kill -9 4399 try to restart your daemon and check the port $> sudo netstat -lntp|grep 22000

alexm · ‎07-12-2018

Syncronizing data between clusters can be accomplished via distcp, BDR, or ingesting data into both clusters simulatenously using 3rd party tools. The best tool depends on your use case, risk tolerance, and budget. We don't recommend spanning clusters across large geographic regions (e.g. US to EU); network latency and bandwidth are usually not suitable and could easily result in the slow query times you're experiencing. We DO support spanning clusters across AWS Availability Zones if certain conditions are met; see Appendix A of Cloudera Enterprise Reference Architecture for AWS Deployments (PDF) details. For comparison, the latency between AWS AZs is typically sub-millisecond. Spanning bare metal clusters across multiple data centers will be addressed in the next release of Cloudera Enterprise Reference Architecture for Bare Metal Deployments (PDF), to coincide with C6. It will look similar to the AWS guidance, but with the additional caveat that network latency between sides should not exceed 10ms. Kudu does not support rack awareness. Not all services provide HA.

MattS · ‎05-01-2018

The reason you were seeing HdfsParquetTableWriter::ColumnWriter is that I was testing the bug using the syntax: CREATE TABLE db.newTable STORED AS PARQUET AS SELECT a.topLevelField, b.priceFromNestedField FROM db.table a LEFT JOIN a.nestedField This was purely to force the bug to occur - if you just did the SELECT in Hue it would often succeed because it only brings back the first 100 rows - to consistently trigger the crash I had to make Impala read from both Parquet files. No other query was running at the time. Anyway, as Chris says, the bug appears to be fixed in 5.14.2. The job which originally consistently triggered the crash has now been running unchanged over the same source data for 20 hours without hitch. Thanks for your help Matt

AcharkiMed · ‎04-19-2018

Take a look to this: http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-ODBC-JDBC-bad-performance-rows-fetch-is-very-slow-from-a/m-p/61152#M3751 Good luck.

epowell · ‎03-15-2018

Thank you, @Lars Volker. Definitely learned something new there!

Lars Volker · ‎12-10-2017

Hi Davood, Impala needs a column type for column3 and NULL does not allow the planner to infer the type. Using a cast to specify the type will work: create table v as select i, cast(null as int) as j from t; Cheers, Lars

Lars Volker · ‎11-14-2017

Hi mauricio, Impala currently does not support graceful node decommissioning. We're tracking work on this feature in IMPALA-1760, but we currently are not targeting it for a particular release. Unfortunately that only leaves the option of killing the daemon. Cheers, Lars

yehudaks · ‎11-08-2017

Hi, it seems now that the bottle neck is not impala. I created a table with the query result and it took only few minutes. Currenlty I concetrate with checking network / Tableau configuration Thanks

Lars Volker · ‎11-01-2017

Hi hrishi1dypim, Have you restarted all Impala roles including statestored and catalogd after the upgrade? Cheers, Lars

kakanab · ‎10-20-2017

We have workaround first point is inplace but we have experienced connection establishment issues from webserver to impala with below error. It got fixed itself after couple of hours and would like to know what causes this behavior. [Simba][ImpalaJDBCDriver](500176) Error connecting to HiveServer2, please verify connection details. Please help us to understand what causes this issue. Thank you, Aj

Tim Armstrong · ‎08-24-2017

I'm not sure there are risks specifically. The best practice is to use Cloudera manager to configure memory limits for different services, so this is the right way to configure things. Cloudera Manager does have support to help set up memory limits for applications: https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_autoconfig.html#concept_xjy_vb3_rn . For a production system, it's important to put thought into how much memory your system needs and how it's allocated between different services. E.g. as an earlier poster saw, 256MB is not enough memory to do much interesting with Impala.

ponypony · ‎08-15-2017

Thanks Lars. Shannon

desind · ‎07-13-2017

@csguna 2.6.32-573.22.1.el6.x86_64 Redhat 6.7

Lars Volker · ‎06-20-2017

Steps to generate SSL certificates are generally independent of Impala and well documented. One such place where you can find more information is here: https://www.digitalocean.com/community/tutorials/openssl-essentials-working-with-ssl-certificates-private-keys-and-csrs

ponypony · ‎06-14-2017

Ah, thanks.

thewayofthinkin · ‎04-24-2017

Yeap, you're right

cjervis · ‎04-11-2017

I moved it for you. 🙂

imad87 · ‎04-03-2017

Your comment gave me the clue, when I generated the script, I missed the statment that follows: ROW FORMAT DELIMITED, namely, -FIELDS TERMINATED BY ','. So the correct create statement would be: CREATE EXTERNAL TABLE IF NOT EXISTS ccce_apl( APL_LNK INT, UPDT_DTTM CHAR(26), UPDT_USER CHAR(8), RLS_ORDR_MOD_CD CHAR(12), RLS_ORDR_MOD_TXT VARCHAR(255) ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE location '/hdfs/data-lake/master/criminal/csv/ccce_apl'; Thanks.

alex.behm · ‎03-31-2017

Thanks for following up! Glad to hear you could squeeze out some performance.

Lars Volker · ‎03-28-2017

Cool, glad you found the issue.

Lars Volker · ‎03-27-2017

Cool, I'm glad you found an answer. Thanks for sharing it here, too. Cheers, Lars

Lars Volker · ‎03-15-2017

Thank you for the additional information. The team working on the driver has identified the issue and a fix will be included in the next release of the drivers. Until then I'm afraid you'll be stuck with the somewhat ugly workaround. Cheers, Lars

Antoni · ‎03-10-2017

Thanks, To clarify, the proxy protocol is not HTTP related. IT works on TCP level (http://www.haproxy.org/download/1.8/doc/proxy-protocol.txt) but it does require that client and server be compatible with it.

Lars Volker · ‎02-22-2017

I just saw this thread after commenting on the Jira. Would "conv()" be a suitable workaround here? select conv('100010', 2, 10); +-----------------------+ | conv('100010', 2, 10) | +-----------------------+ | 34 | +-----------------------+ Fetched 1 row(s) in 0.24s More information on conv() can be found in the Impala documentation. Edit: To make things complete, the Jira is IMPALA-4968.

AnisurRehman · ‎02-03-2017

Glad to see advice and I have posted my query over there. Regards

alex.behm · ‎02-02-2017

@gaurang would you be open to sharing your CREATE TABLEs, CREATE VIEW and the query that has slow planning time? No need for the data, just that should be sufficient for us to understand better what's going on. Like Lars said, you are probably hitting IMPALA-4242 which explains the slow equivalence class computation, but I'd also like to understand the slow single-node planning time. Thanks!

Online	Offline
Last Visited	‎09-24-2019 05:29 PM

Date Registered	‎12-07-2015 10:33 AM
Last Visited	‎09-24-2019 05:29 PM
Total Messages Posted	83
Total Kudos Received	23

Re: When I add a new rack some Impala queries beca...

Re: Create table as select issue Unsupported type...

Re: How to gracefully stop an impalad?

Re: Need help with Impala 2.8 on CDH 5.10 upgrade

Re: Inverse of function bin

Re: combine small parquet files

Re: Impalad COORDINATOR_ONLY with HDFS Short-Circu...

Re: Impala queries through impala JDBC is running ...

Re: Accessing Impala views using Spring JDBC

Re: impala error - ERROR: ExecPlanRequest rpc quer...

Re: When I add a new rack some Impala queries beca...

Re: Impala critical bug in CDH 5.14.0

Re: Impala JDBC 10x Slower Vs. Shell

Re: 'ascii' codec can't encode character u'\xe8' i...

Re: Create table as select issue Unsupported type...

Re: How to gracefully stop an impalad?

Re: Question about query performance in Impala

Re: Error deserializing item TABLE on impalad vers...

Re: IMPALA JDBC connection

Re: "Memory Limit Exceeded" error on Impala when i...

Re: Partition, file size and number of files quest...

Re: Impala Catalogue server down after upgrading f...

Re: Steps to generate ssl certificate for Impala

Re: How to outer join child array

Re: Need help with Impala 2.8 on CDH 5.10 upgrade

Re: Receiving ZooKeeperException when trying conne...

Re: Create Hive Table from HDFS files

Re: Impala slow aggregation

Re: Use impala JDBC in Java

Re: How to get header in Impala output csv file

Re: Impal Kudu JDBC Delete on PK not working ([Sim...

Re: How to keep original IP address when using HA ...

Re: Inverse of function bin

Re: Need clarity on sqoop import --impala-import

Re: When querying a VIEW, query planning takes a l...