If you're not using Tez UI to debug your Hive queries you need to start today. Ambari installs Tez UI by default, but some people need a standalone option. Tez UI is easy to install and can go anywhere, meaning there's no reason for you to wonder why that Hive query won't run faster.
Why You Should Use Tez UI:
Whether in Ambari or Standalone, here's some of the things you're missing if you don't use Tez UI:
1. Inventory of all queries executed, with search and filter.
2. Drill into a query and see the actual query text. Great if a BI tool is generating long-running queries.
3. Vertex Swimlane. This one is new in HDP 2.5 but it helps you quickly pinpoint the long-running parts of your query and optimize them.
How To Install Tez UI Standalone:
The Tez UI Installation Instructions give the details on installing Tez UI, but it can be a bit hard to put the pieces together. In this How-To we will cover installing Tez UI into Apache httpd on an HDP 2.5 installation. The installation instructions don't change much for different web servers or for app servers like Tomcat.
Installing the UI in Apache httpd assuming a CentOS base OS.
- Install Tez and httpd: yum install httpd tez
- Create a directory for the UI: mkdir /var/www/html/tez-ui
- Change into this new directory: cd /var/www/html/tez-ui
- Extract the Tez UI WAR into this new directory: unzip /usr/hdp/current/tez-client/ui/tez-ui-0.7.0.2.5.0.0-1245.war
- Open /var/www/html/tez-ui/scripts/configs.js in a text editor and modify these values.
- Set timelineBaseUrl to your Application Timeline Server (ATS) endpoint. Usually something like: http://ats.example.com:8188
- Set RMWebUrl to your Resource Manager (RM) endpoint. Usually something like http://rm.example.com:8088
Configuring Tez:
You need to tell Tez to log its history to ATS using the logging service class. Also you should tell Tez where the UI is located, so it can cross link to the Tez UI from within the Resource Manager UI. All together these are needed in your tez-site.xml:
<property> <description>Enable Tez to use the Timeline Server for History Logging</description> <name>tez.history.logging.service.class</name> <value>org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService</value> </property> <property> <description>URL for where the Tez UI is hosted</description> <name>tez.tez-ui.history-url.base</name> <value>http://httpd.example.com:80/tez-ui/</value> </property>
That's it. Now you can load Tez UI in your browser.
Example:
Here's an example configs.env and screenshot of the resulting UI. Notice that timeline and rm have both been modified from the original:
[vagrant@hdp250 scripts]$ pwd /var/www/html/tez-ui/config [vagrant@hdp250 scripts]$ cat configs.env /** * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information * regarding copyright ownership. The ASF licenses this file * to you under the Apache License, Version 2.0 (the * "License"); you may not use this file except in compliance * with the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ ENV = { hosts: { /* * Timeline Server Address: * By default TEZ UI looks for timeline server at http://localhost:8188, uncomment and change * the following value for pointing to a different address. */ timeline: "http://hdp250.example.com:8188", /* * Resource Manager Address: * By default RM REST APIs are expected to be at http://localhost:8088, uncomment and change * the following value to point to a different address. */ rm: "http://hdp250.example.com:8088", /* * Resource Manager Web Proxy Address: * Optional - By default, value configured as RM host will be taken as proxy address * Use this configuration when RM web proxy is configured at a different address than RM. */ //rmProxy: "http://localhost:8088", }, /* * Time Zone in which dates are displayed in the UI: * If not set, local time zone will be used. * Refer http://momentjs.com/timezone/docs/ for valid entries. */ //timeZone: "UTC", /* * yarnProtocol: * If specified, this protocol would be used to construct node manager log links. * Possible values: http, https * Default value: If not specified, protocol of hosts.rm will be used */ //yarnProtocol: "<value>", };
This configuration was used to generate the first screenshot in this document, above.
CORS:
On a multi-node cluster, chances are you will need to enable CORS in ATS to properly view data. If you connect and see an error about CORS not enabled, see the Apache Documentation for the relevant configurations you will need to set, most importantly you will need to enable yarn.timeline-service.http-cross-origin.enabled
Note On Secure Clusters:
On secure clusters your client web browser will need to be configured to talk to the Kerberized cluster. Depending on your browser you may need to kinit your client or use a plugin. A good way to confirm connectivity is to first see if your browser can connect to the Resource Manager UI. If yes, you should also be able to connect to the standalone Tez UI.
