Member since
08-20-2016
4
Posts
0
Kudos Received
0
Solutions
08-23-2016
03:23 AM
@Sunile Manjee There's no .hiverc in the directory. I tried with an empty initialize file with -i and yet, predicate pushdown works fine with hive-cli but it doesn't work with oozie.
... View more
08-22-2016
11:26 AM
Okay, there are a few properties like `hive.optimize.point.lookup` that are being set in oozie but are undefined in hive-cli. I think that means there is a version mis-match. But all the jars in oozie share lib have the same version (1.2.1).
... View more
08-21-2016
05:13 AM
Hi @Sunile Manjee, thanks for the response. I expect .hiverc to be available in hive conf directory but couldn't find it there. Where can I find it?
Also, does this mean that the output of `set -v;` for cli and oozie action will differ? I have the output of this command for cli and oozie but I couldn't find any property that was different and relevant to predicate push down. Would you like me to share it if that helps?
... View more
08-20-2016
04:57 AM
Hi, I have a partitioned hive table that has multiple partition columns. My query is of the form: select audf(column1) , sum(column2) from table1 join table2 on (table1.id = table2.id) where ((table1.partition_column1 = 2016 and table1.partition_column2 = 08 and table1.partition_column3 = 09) or (table1.partition_column1 = 2016 and table1.partition_column2 = 08 and table1.partition_column3 = 10)) and ((table2.partition_column1 = 2016 and table2.partition_column2 = 08 and table2.partition_column3 = 09) or (table2.partition_column1 = 2016 and table2.partition_column2 = 08 and table2.partition_column3 = 10)) group by audf(column1); Predicate push down happens when this query is run through hive-cli and input splits that are added are significantly low in number as compared to when this query is run as part of oozie hive action. All the 4 configuration properties related to ppd are set to true in both, hive as well as oozie configurations. I am using HDP 2.4.
... View more
Labels: