Member since
05-03-2016
12
Posts
3
Kudos Received
0
Solutions
03-21-2017
05:55 PM
Hi @Eugene Koifman Just Wondering if hive.mapred.mode=strict , why hive not using distribute by sort by Limit to replace the order by execution plan? i have tested with my data , it seems those two query are identical on final result; @mqureshi Thank you very much , in general distribute by sort by is not the same as order by , but if we had a limit after it , hive will put another reducer to make the final result. Thank you very much
... View more
03-06-2017
07:04 PM
@mqureshi Thanks for your answer. turn off the nonstrict does offer some speed to the query. @Eugene Koifman The most interesting thing i found was setting the hive.map.aggr = false . it reduced the query speed from 15mins down to 3 mins .
... View more
03-06-2017
02:06 AM
Thanks @mqureshi Does DISTRIBUTE BY (order by column), SORT BY (sort column). always give the same result as using ORDER BY (order by column)? Order by is a total order and distributed by -> sort by could give the total order as well ? because we have an LIMIT clause at the end.
... View more
03-05-2017
04:25 PM
Hi Is there any way to speed up the Hive Order by With Limit Query in any way? suppose the base table contains 1Km (1,000,000,000) rows and now perform the query Select * from Table t order by t.order_num Limit 10000; no table and query modification allowed since the real query are much complicated in the Select part and a pre-sorted/Orc Partitioned tables does not helps much. since all rows goes to the same reducer to get the correct order, it greatly impacted the performance of the Hive on Tez performance. And we are in an POC to show off The power of Hive on Tez but got stuck in this order by query. Thank you very much in advance
... View more
Labels:
- Labels:
-
Apache Hive
01-07-2017
05:38 AM
Thank you very much
... View more
08-22-2016
04:18 AM
1 Kudo
Just Wondering if anyone could provide a deep dive on current Apache
Storm 1.x's ? there is nowhere to find a good explaination on those
topics. Much Appreciated in advance .
... View more
Labels:
- Labels:
-
Apache Storm
05-30-2016
07:00 AM
cloudbreak deployer with openstack: when create cluster, error "org.openstack4j.api.exceptions.ConnectionException: java.net.UnknownHostException". My cloud is using version openstack kilo, which is distributed by 2 physical servers, cloudbreak deployer version is 1.2.1, when deployer is started, associating it with a floating-ip, and then ssh it, excute cbd init and cbd start commands, then successfully access and login in to http://deployer-floating-ip:3000 via browser, then I create credential、network and so on, and then I create a cluster with a bluepoint, but deployer reports and error "org.openstack4j.api.exceptions.ConnectionException: java.net.UnknownHostException: controller". When I create credential, I cannot use keystone v2, I have to use key stone v3, because keystone only support v3。
... View more
Labels:
- Labels:
-
Hortonworks Cloudbreak