Cloudera Community
Tim Armstrong
user rank
user rank
Guru

Re: Impala retries even when "SetError(const char*...

by Guru Guru in Support Questions
‎09-29-2017 05:21 PM
‎09-29-2017 05:21 PM
I just saw this in my email backlog. Yes this is the expected behaviour. Your UDF may be called again on the same thread or a different thread. After you call SetError() the query will fail but the error will take some time to propagate. ... View more

Re: Impala query does not respond,seems stuck or g...

by Explorer in Support Questions
‎09-26-2017 12:46 AM
‎09-26-2017 12:46 AM
have seen the execution summary ,its taking 15 mins and after that query is getting cancelled.have already done compute stats for involved tables. there is no memory problem as it shows consumption 340 MB out of 35 GB of memory. ... View more

Re: Impala - Timestamp - order by works?

by Former Member in Support Questions
‎09-25-2017 07:20 PM
‎09-25-2017 07:20 PM
Hi,     Hbase table   table in hbase   hbid is combination of 3 columns. two string and one timestamp column anhdu6jbc#cpu#2017-08-01 00:00:09.0= hostname#matrix__timestamp hbid- anhdu6jbc#cpu__2017-08-01 00:00:09.0 country-string-addf matrix_value- double-8.9   select extract(split_part(hbid,'_',3),"day") as Day,split_part(split_part(hbid,'_',2),'#',1) as Hostname,split_part(hbid,'_',3) as starttime,matrix_value from table where split_part(split_part(hbid,'_',2),'#',2)='cpu'and matrix_value <=100 and split_part(split_part(hbid,'_',2),'#',1)='2jdnca' and (to_date(split_part(hbid,'_',3)) >= '2017-08-01' and to_date(split_part(hbid,'_',3)) <= '2017-08-08') order by matrix_value limit 10   Exception   RetriesExhaustedException: Failed after attempts=3, exceptions: Tue Sep 26 10:17:39 SGT 2017, RpcRetryingCaller{globalStartTime=1506392256600, pause=100, retries=3}, java.io.IOException: Call to x01shdpeapp4a.sgp.dbs.com/10.92.139.146:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1483400, waitTime=3001, operationTimeout=3000 expired. Tue Sep 26 10:17:42 SGT 2017, RpcRetryingCaller{globalStartTime=1506392256600, pause=100, retries=3}, java.io.IOException: Call to x01shdpeapp4a.sgp.dbs.com/10.92.139.146:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1483402, waitTime=3001, operationTimeout=3000 expired. Tue Sep 26 10:17:46 SGT 2017, RpcRetryingCaller{globalStartTime=1506392256600, pause=100, retries=3}, java.io.IOException: Call to x01shdpeapp4a.sgp.dbs.com/10.92.139.146:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1483404, waitTime=3001, operationTimeout=3000 expired. CAUSED BY: IOException: Call to x01shdpeapp4a.sgp.dbs.com/10.92.139.146:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1483404, waitTime=3001, operationTimeout=3000 expired. CAUSED BY: CallTimeoutException: Call id=1483404, waitTime=3001, operationTimeout=3000 expired. select extract(split_part(hbid,'_',3),"day") as Day,split_part(split_part(hbid,'_',2),'#',1) as Hostname,split_part(hbid,'_',3) as starttime,matrix_value from tablename where split_part(split_part(hbid,'_',2),'#',2)='cpu'and matrix_value <=100 and split_part(split_part(hbid,'_',2),'#',1)='shdjd1d' order by trunc(split_part(hbid,'_',3),'DD')     Exception   RetriesExhaustedException: Failed after attempts=3, exceptions: Tue Sep 26 10:07:33 SGT 2017, RpcRetryingCaller{globalStartTime=1506391650523, pause=100, retries=3}, java.io.IOException: Call to x01shdpeapp4a.sgp.dbs.com/10.92.139.146:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1483303, waitTime=3001, operationTimeout=3000 expired. Tue Sep 26 10:07:36 SGT 2017, RpcRetryingCaller{globalStartTime=1506391650523, pause=100, retries=3}, java.io.IOException: Call to x01shdpeapp4a.sgp.dbs.com/10.92.139.146:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1483305, waitTime=3001, operationTimeout=3000 expired. Tue Sep 26 10:07:40 SGT 2017, RpcRetryingCaller{globalStartTime=1506391650523, pause=100, retries=3}, java.io.IOException: Call to x01shdpeapp4a.sgp.dbs.com/10.92.139.146:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1483307, waitTime=3001, operationTimeout=3000 expired. CAUSED BY: IOException: Call to x01shdpeapp4a.sgp.dbs.com/10.92.139.146:60020 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=1483307, waitTime=3001, operationTimeout=3000 expired. CAUSED BY: CallTimeoutException: Call id=1483307, waitTime=3001, operationTimeout=3000 expired. ... View more

Re: Impala Thrift Server

by Contributor in Support Questions
‎09-21-2017 04:56 AM
‎09-21-2017 04:56 AM
Before your reply I've rebooted the node after that it's run well.   Now I can see some issue(Port running on other Node) again when I'm running the  COMPUTE STATS command on Impala.   COMPUTE STATS: 1.I have a table with old Parquet data. 2.Now I have added parquet data with New data types(INT TO STRING) for the same columns. 3.I have created new table in the same location with new schema (Impala > CREATE EXTERNAL TABLE database.table2 like parquet '/home/output/university/client=england/campaign=second/details=students/part-r-00111-5fce6c4d-784e-457f-9a01-aa6d6ec1187c.snappy.parquet'; Impala > SHOW CREATE TABLE table2; Then Ive created tablw with table2 schema) 4. /home/output/university/client=england/campaign=second/details=students/part-r-00111-5fce6c4d-784e-457f-9a01-aa6d6ec1187c.snappy.parquet' has an incompatible Parquet schema for column 'mobistat.psrvoicecdma.systemid'. Column type: STRING, Parquet schema: optional int32 systemid [i:22 d:1 r:0]     please help me out in this.     ... View more

Re: Getting Error : Spilling has been disabled due...

by Guru Guru in Support Questions
‎09-20-2017 12:19 AM
1 Kudo
‎09-20-2017 12:19 AM
1 Kudo
If you're starting Impala from the command line like that you can configure  flags and environment variables with the /etc/default/impala -  https://www.cloudera.com/documentation/enterprise/latest/topics/impala_processes.html#starting_via_cmdline .  The relevant variable in that file is IMPALA_SERVER_ARGS.   (If anyone else reads this, if you're using Cloudera Manager you can configure the scratch directories through the UI. You probably won't have to since CM does a pretty good job of autoconfiguring scratch directories). ... View more

Re: "Memory Limit Exceeded" error on Impala when i...

by Guru Guru in Support Questions
‎08-24-2017 08:32 AM
‎08-24-2017 08:32 AM
I'm not sure there are risks specifically. The best practice is to use Cloudera manager to configure memory limits for different services, so this is the right way to configure things.   Cloudera Manager does have support to help set up memory limits for applications: https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_autoconfig.html#concept_xjy_vb3_rn .    For a production system, it's important to put thought into how much memory your system needs and how it's allocated between different services.  E.g. as an earlier poster saw, 256MB is not enough memory to do much interesting with Impala.     ... View more

Re: Impala running slow after upgrade

by Contributor in Support Questions
‎08-16-2017 03:55 PM
‎08-16-2017 03:55 PM
Agent and edge node where HUE was running was not upgraded  ... View more

Re: impala query memory limit

by Contributor in Support Questions
‎08-14-2017 02:02 PM
‎08-14-2017 02:02 PM
Hi Tim,   Thanks for the information.   A minor observation:   I found that "tcmalloc_physical_bytes_reserved_across_impalads" and "mem_tracker_process_limit_across_impalads" are for v5.3.x (ref: https://www.cloudera.com/documentation/enterprise/5-3-x/topics/cm_metrics_impala.html) and "tcmalloc_physical_bytes_reserved" and "mem_tracker_process_limit" are for v5.7.x and above (https://www.cloudera.com/documentation/enterprise/5-10-x/topics/cm_metrics_impala_daemon.html). I'm using v5.10, however, when I try to do the Chart Build in CM I can only find the previous 2 metrics though (ie, "tcmalloc_physical_bytes_reserved_across_impalads" and "mem_tracker_process_limit_across_impalads" ).    Also from the CM rest api, it seems like those 2 sets of metrics are the same but just different data aggregation format in the returned json?   Regards, S.   ... View more

Re: Impala max/min slow

by Contributor in Support Questions
‎08-11-2017 08:05 PM
‎08-11-2017 08:05 PM
Thanks, i will give it a try, i can create my final tables to have it converted to timestamp.   Shannon ... View more

Re: Should Impala release memory after use?

by Expert Contributor in Support Questions
‎08-11-2017 07:57 AM
‎08-11-2017 07:57 AM
Thank you for the detailed answer ... View more
Contact Me
Online
Offline
Last Visited
‎02-11-2021 06:07 PM
Community Statistics
Member Since ‎07-29-2015 04:07 PM
Last Visited ‎02-11-2021 06:07 PM
Posts 535
Kudos received 141