Viewing logs for Hive query Executions

[#1]

I'm quite new to HDP, and still I'm learning about this platform.

I'm following the SensorData tutorial and I wanted to know if it's possible to see the logs for the Map Reduce executions associated with particular Hive Query.

SELECT truckid, avg(mpg) avgmpg FROM truck_mileage GROUP BY truckid;

I'm running the above query, and I'm trying to see all the underlying MapReduce jobs.

How to get this info?

[#2]

@Amila De Silva

Please check under /var/log/hive. Log for all other engines can also be similarly found under /var/log/<hdfs,spark,hbase etc>

[#3]

@mqureshi

Thanks for the prompt reply. I checked that but the only content I found in that file was

WARNING: Use "yarn jar" to launch YARN applications.

When going through this tutorial, I found that by setting execution.engine to mr, you can get all the logs for a MR job, which is what I was looking for. Thanks for the help.

[#4]

@Amila De Silva

If you are looking for application logs:

yarn logs -applicationId <application ID>

Example:

yarn logs -applicationId application_1470266735999_002

[#5]

@Amila De Silva

There is a simple way to do this. Go to yarn through ambari. Click on yarn then click on quick links, then resource manager UI. then click on job name. and you have full access to all logs.

[#6]

@ mqureshi

Go to Resource Manager UI: http://127.0.0.1:8088/cluster, click on your application_... job and then on the Attempt ID line click on Logs. You may also want to use your Tez View in Ambari http://127.0.0.1:8080/#/main/views/TEZ