Support Questions
Find answers, ask questions, and share your expertise

Debugging Pig UDFs on Tez

I have a Pig UDF which runs great on MR but fails on Tez. I added logs using EvalFunc.getLogger() and also tried System.out.println(). When I run Pig on MR I can find all my log printouts in files obtained using "yarn logs", however when I run the script on Tez, it is producing no log output. I also tried to add "-d DEBUG" but in vain. Any ideas how to produce logs when Pig runs on Tez? [This is HDP-2.3.4 and Pig-0.15]


Re: Debugging Pig UDFs on Tez


First make sure you have Tez job application id, and then just yarn logs tool.

To customize Tez log level, you need to set '' and 'tez.task.log.level'.

Also, you may want to wait for Tez job finished(check yarn web console), because pig may keep a Tez session for a while and log of a running application may be incomplete.

Re: Debugging Pig UDFs on Tez

Thanks for your reply, I'll try it next week. Re tez.*.log.level, I see in the docs that the default is INFO, and I logged my messages using, but will try to set it explicitly to INFO or DEBUG. I'll also try to get logs with some delay, after a cup of coffee 🙂

Re: Debugging Pig UDFs on Tez

That didn't work. I set both tez.*.log.level to DEBUG, and used but could find only output from the calls on the frontend, and nothing from the backend. Also waited for a while but to no avail.

Re: Debugging Pig UDFs on Tez

Super Guru

Use the Ambari Pig View, it's much better for running and debugging Pig scripts

Re: Debugging Pig UDFs on Tez

Not applicable, since the views are not installed, and the team doesn't have admin permissions.

Re: Debugging Pig UDFs on Tez


Heres a pretty extensive list of options to debug, including Tez and options to run with PigUnit unit testing utility