- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
slow query in hive
- Labels:
-
Apache Hive
Created ‎06-01-2016 09:34 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi:
actually i have this query very slow, any suggestion to improve that?
select codinterno,sum(freq) as freq,codtxf from journey_oficina_v4 where codnrbeenf = '3008' and (fechaoprcnf >= '2016-04-01 00:00:00.0' and fechaoprcnf < '2016-04-02 00:00:00.0' ) and codtxf = 'STR01OON' group by codinterno, codtxf order by freq desc
i just have a bucket for codnrbeenf and is not in the query so....
Thanks
Created ‎06-01-2016 10:09 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
as you are using Transactional Table you can not take advantage of sort by on fechaoprcnf column.Apart from partitioning try to create storage index on the table using tblproperties ("orc.create.index"="true","orc.compress"="ZLIB", "orc.stripe.size"="268435456", "orc.row.index.stride"="10000")
-- orc stripe and index.stride value in this case are default try to tune these value and compare performace results.
Created ‎06-01-2016 12:14 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi:
you are right i have received this error:
Error: Error while compiling statement: FAILED: SemanticException [Error 10298]: ACID insert, update, delete not supported on tables that are sorted, table journey_oficina (state=42000,code=10298)
so, I have removed the "transactional"="true", and the statement was ok, but now i cant make ACID operaitons, also it have improve the query, but now HOW CAN I UPDATE DE TABLE??
Created ‎06-02-2016 12:21 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi:
whaMany thanks i create that and it faster now, but i cant do ACID operation, so, what can i do??
CREATE EXTERNAL TABLE IF NOT EXISTS journey_oficina( FECHAOPRCNF timestamp, codnrbeenf string, CODINTERNO string, CODTXF string, FREQ BIGINT ) CLUSTERED BY (codnrbeenf) SORTED BY (FECHAOPRCNF) INTO 60 BUCKETS stored as ORC LOCATION '/RSI/tables/logs/journey_oficina' TBLPROPERTIES ("immutable"="false", "orc.create.index"="true","orc.compress"="ZLIB", "orc.stripe.size"="67108864","orc.row.index.stride"="25000");
thanks
Created ‎06-02-2016 12:29 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
good to see that worked but due to limitation with Transactional table you can not use sorted by, may be in coming future version it will be supported.
Created ‎06-03-2016 05:42 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
so, this month it will be in production, so what happend if the diary batch fail this day???
Created ‎06-05-2016 09:29 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I could not get your question here, could you please elobarate what you are asking?
Created ‎06-06-2016 03:18 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I mean if i need to change into the table, i need to delete everything and insert again??
Created ‎06-07-2016 05:38 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created ‎06-08-2016 06:26 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
it seems that all your queries has been answered, could you please spare some time and accept a best answer in this thread.

- « Previous
-
- 1
- 2
- Next »