Enable/disable Tez and verify

kaliyugantagoni — Sun, 18 Aug 2019 13:03:21 GMT

Stack : Installed HDP-2.3.2.0-2950 using Ambari 2.1.

Tables from a SQL Server schema have been imported on HDFS using Sqoop.

The Hive config in Ambari is as follows(Tez is the execution engine) :

There is a huge table with 24187713416 rows but it is just an ordinary external table with no partitioning etc., when I execute a select query, following is the output(I couldn't figure out if Tez is in action) :

hive (odp_dw_may2016)>
                     >
                     > select * from FactSampleValue where Snapshot_Id = 643067 and SmapiName_ver = 'Abnormal_ShutDown_ct_1' and SmapiColName = 'Abnormal_ShutDown_ct_1_000';
Query ID = hive_20160516092942_4b1c90ab-77bd-4ea8-bf3e-9af611a63143
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1446726117927_0076)
--------------------------------------------------------------------------------
        VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
--------------------------------------------------------------------------------
Map 1 ..........   SUCCEEDED   3098       3098        0        0       0       0
--------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 1341.80 s
--------------------------------------------------------------------------------
OK
factsamplevalue.chassino        factsamplevalue.snapshot_id     factsamplevalue.ecu_id  factsamplevalue.smapiname_ver   factsamplevalue.smapicolname    factsamplevalue.indexet factsamplevalue.value        factsamplevalue.samplefrequency factsamplevalue.vectorperc      factsamplevalue.smapiname       factsamplevalue.diffvalue       factsamplevalue.commvalue
1872663 643067  28      Abnormal_ShutDown_ct_1  Abnormal_ShutDown_ct_1_000      0       1.0     NULL    100.0   Abnormal_ShutDown_ct    NULL    NULL
Time taken: 1350.861 seconds, Fetched: 1 row(s)

I have several questions pertaining only to the usage of Tez :

Can I safely assume that Tez is the execution engine right now ? Does Tez maintain its logs where the Hive query execution is logged ?
I read in the Hortonworks doc. about enabling Tez, it is necessary even when Ambari is used ?
Can the change in the Hive config via Ambari seamlessly switch between Tez and MapReduce ?
I checked the Apache Tez doc. but I am unclear about the utility of the Tez UI ?

Re: Enable/disable Tez and verify

bleonhardi — Mon, 16 May 2016 18:14:35 GMT

1) yes you can see the "Tez session was closed ...

2) In anything after HDP2 tez is enabled by default. MapReduce might be going away as an option anyway

3) You can still use set execution engine in queries set hive.execution.engine=mr or tez

4) Not sure what you mean with utiliy. The Tez view in ambari would provide the functionality I am not completely sure about the out of the box integration with resource manager

https://www.youtube.com/watch?v=xyqct59LxLY

question Re: Enable/disable Tez and verify in Archives of Support Questions (Read Only)

Enable/disable Tez and verify

Re: Enable/disable Tez and verify