- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Enable/disable Tez and verify
- Labels:
-
Apache Hive
-
Apache Tez
Created on ‎05-16-2016 10:15 AM - edited ‎08-18-2019 06:03 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Stack : Installed HDP-2.3.2.0-2950 using Ambari 2.1.
Tables from a SQL Server schema have been imported on HDFS using Sqoop.
The Hive config in Ambari is as follows(Tez is the execution engine) :
There is a huge table with 24187713416 rows but it is just an ordinary external table with no partitioning etc., when I execute a select query, following is the output(I couldn't figure out if Tez is in action) :
hive (odp_dw_may2016)> > > select * from FactSampleValue where Snapshot_Id = 643067 and SmapiName_ver = 'Abnormal_ShutDown_ct_1' and SmapiColName = 'Abnormal_ShutDown_ct_1_000'; Query ID = hive_20160516092942_4b1c90ab-77bd-4ea8-bf3e-9af611a63143 Total jobs = 1 Launching Job 1 out of 1 Tez session was closed. Reopening... Session re-established. Status: Running (Executing on YARN cluster with App id application_1446726117927_0076) -------------------------------------------------------------------------------- VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED -------------------------------------------------------------------------------- Map 1 .......... SUCCEEDED 3098 3098 0 0 0 0 -------------------------------------------------------------------------------- VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 1341.80 s -------------------------------------------------------------------------------- OK factsamplevalue.chassino factsamplevalue.snapshot_id factsamplevalue.ecu_id factsamplevalue.smapiname_ver factsamplevalue.smapicolname factsamplevalue.indexet factsamplevalue.value factsamplevalue.samplefrequency factsamplevalue.vectorperc factsamplevalue.smapiname factsamplevalue.diffvalue factsamplevalue.commvalue 1872663 643067 28 Abnormal_ShutDown_ct_1 Abnormal_ShutDown_ct_1_000 0 1.0 NULL 100.0 Abnormal_ShutDown_ct NULL NULL Time taken: 1350.861 seconds, Fetched: 1 row(s)
I have several questions pertaining only to the usage of Tez :
- Can I safely assume that Tez is the execution engine right now ? Does Tez maintain its logs where the Hive query execution is logged ?
- I read in the Hortonworks doc. about enabling Tez, it is necessary even when Ambari is used ?
- Can the change in the Hive config via Ambari seamlessly switch between Tez and MapReduce ?
- I checked the Apache Tez doc. but I am unclear about the utility of the Tez UI ?
Created ‎05-16-2016 11:14 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
1) yes you can see the "Tez session was closed ...
2) In anything after HDP2 tez is enabled by default. MapReduce might be going away as an option anyway
3) You can still use set execution engine in queries set hive.execution.engine=mr or tez
4) Not sure what you mean with utiliy. The Tez view in ambari would provide the functionality I am not completely sure about the out of the box integration with resource manager
Created ‎05-16-2016 11:14 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
1) yes you can see the "Tez session was closed ...
2) In anything after HDP2 tez is enabled by default. MapReduce might be going away as an option anyway
3) You can still use set execution engine in queries set hive.execution.engine=mr or tez
4) Not sure what you mean with utiliy. The Tez view in ambari would provide the functionality I am not completely sure about the out of the box integration with resource manager
