Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Enable/disable Tez and verify

avatar
Super Collaborator

Stack : Installed HDP-2.3.2.0-2950 using Ambari 2.1.

Tables from a SQL Server schema have been imported on HDFS using Sqoop.

The Hive config in Ambari is as follows(Tez is the execution engine) :

4232-ambari-config-execution-engine-tez.png

There is a huge table with 24187713416 rows but it is just an ordinary external table with no partitioning etc., when I execute a select query, following is the output(I couldn't figure out if Tez is in action) :

hive (odp_dw_may2016)>
                     >
                     > select * from FactSampleValue where Snapshot_Id = 643067 and SmapiName_ver = 'Abnormal_ShutDown_ct_1' and SmapiColName = 'Abnormal_ShutDown_ct_1_000';
Query ID = hive_20160516092942_4b1c90ab-77bd-4ea8-bf3e-9af611a63143
Total jobs = 1
Launching Job 1 out of 1
Tez session was closed. Reopening...
Session re-established.
Status: Running (Executing on YARN cluster with App id application_1446726117927_0076)
--------------------------------------------------------------------------------
        VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
--------------------------------------------------------------------------------
Map 1 ..........   SUCCEEDED   3098       3098        0        0       0       0
--------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 1341.80 s
--------------------------------------------------------------------------------
OK
factsamplevalue.chassino        factsamplevalue.snapshot_id     factsamplevalue.ecu_id  factsamplevalue.smapiname_ver   factsamplevalue.smapicolname    factsamplevalue.indexet factsamplevalue.value        factsamplevalue.samplefrequency factsamplevalue.vectorperc      factsamplevalue.smapiname       factsamplevalue.diffvalue       factsamplevalue.commvalue
1872663 643067  28      Abnormal_ShutDown_ct_1  Abnormal_ShutDown_ct_1_000      0       1.0     NULL    100.0   Abnormal_ShutDown_ct    NULL    NULL
Time taken: 1350.861 seconds, Fetched: 1 row(s)

I have several questions pertaining only to the usage of Tez :

  1. Can I safely assume that Tez is the execution engine right now ? Does Tez maintain its logs where the Hive query execution is logged ?
  2. I read in the Hortonworks doc. about enabling Tez, it is necessary even when Ambari is used ?
  3. Can the change in the Hive config via Ambari seamlessly switch between Tez and MapReduce ?
  4. I checked the Apache Tez doc. but I am unclear about the utility of the Tez UI ?
1 ACCEPTED SOLUTION

avatar
Master Guru

1) yes you can see the "Tez session was closed ...

2) In anything after HDP2 tez is enabled by default. MapReduce might be going away as an option anyway

3) You can still use set execution engine in queries set hive.execution.engine=mr or tez

4) Not sure what you mean with utiliy. The Tez view in ambari would provide the functionality I am not completely sure about the out of the box integration with resource manager

https://www.youtube.com/watch?v=xyqct59LxLY

View solution in original post

1 REPLY 1

avatar
Master Guru

1) yes you can see the "Tez session was closed ...

2) In anything after HDP2 tez is enabled by default. MapReduce might be going away as an option anyway

3) You can still use set execution engine in queries set hive.execution.engine=mr or tez

4) Not sure what you mean with utiliy. The Tez view in ambari would provide the functionality I am not completely sure about the out of the box integration with resource manager

https://www.youtube.com/watch?v=xyqct59LxLY