Thanks , yes, it worked by setting hive.enforce.bucketing = true. but I didn't understand the background how it affects while querying thru tez and MR? ... View more

@Hari Rongali, Thanks for suggestion, but currently I am not using enforce option. I will try by including with this option and run again. I am using below options, hive.exec.dynamic.partition --> true hive.exec.dynamic.partition.mode --> nonstrict hive.execution.engine --> tez But in the given link, about enforce option, they specifically mentioned, Not needed in Hive 2 .x onward and I am using 2.3.2.0 ... View more

Hi, I checked the some temp table which is input for final table (grdm.fnl_glbl_currency) and it don't have any bucket specified Bucket Columns: [] For those temp table, can see the count from both(MR and TEZ) and count is same. So is it because of bucketing? How it differs from firing the query from TEZ and from MR? What is the solution in such situation. Anything I need to specify, do you have any idea? ... View more

hadoop and hive version - 2.3.2.0-2950 hive> describe formatted grdm.fnl_glbl_currency > ; OK # col_name data_type comment currency_sk int currency_cd varchar(55) currency_dsc varchar(4000) country_nm varchar(255) row_effective_start_dt date row_effective_end_dt date created_job_run_id int updated_job_run_id int created_dts timestamp modified_dts timestamp source_system_join_key varchar(1000) natural_key_col_checksum_str varchar(255) type1_column_checksum_str varchar(255) type2_column_checksum_str varchar(255) system_gen_rec_flg varchar(1) # Partition Information # col_name data_type comment source_system_sk int current_rec_flg string # Detailed Table Information Database: grdm Owner: cmanyar CreateTime: Mon Oct 03 15:51:54 CDT 2016 LastAccessTime: UNKNOWN Protect Mode: None Retention: 0 Location: hdfs://DenaliQA/dl/hive/warehouse/final/be/global/grdm/grdm.db/fnl_glbl_currency Table Type: MANAGED_TABLE Table Parameters: orc.compress SNAPPY transient_lastDdlTime 1475527914 # Storage Information SerDe Library: org.apache.hadoop.hive.ql.io.orc.OrcSerde InputFormat: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat Compressed: No Num Buckets: 10 Bucket Columns: [source_system_join_key] Sort Columns: [] Storage Desc Params: field.delim \u0001 serialization.format \u0001 Time taken: 1.992 seconds, Fetched: 48 row(s) hive> ... View more

Hi, I have uploaded the screenshot here, first one using MR and second using the TEZ. Version 2.3.2.0-2950 ... View more