Support Questions
Find answers, ask questions, and share your expertise

Setting execution type as tez inside pig script.

New Contributor

Is there any way setting up execution mode as tez inside pig script.

i have tried various variation of set exectype=tez to use inside pig script but its not working.

I know it can be done through command line as pig -x tez -f filename

But in the HDPCD certification preparation task, question is asked like

Question: Write a pig script that satsifies following criteria

1. run the pig query using tez as execution engine

2. load data from hive table to pig using hcatalog

So i am confused whether we have to set something inside pig script that will make script run through TEZ.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Setting execution type as tez inside pig script.

Explorer

@pankaj chaturvedi You cannot make Pig run with the Tez execution engine via a Pig script. The "Pig Script" row in the first table of Running Pig with the Tez Execution Engine is wrong.

Instead, look at the same doc link and configure Pig to run with Tez by following the information in the "Command Line" or "Pig Properties" row of the table.

I will file a doc bug to have this fixed.

Let me know if this post does not solve your problem.

View solution in original post

12 REPLIES 12

Re: Setting execution type as tez inside pig script.

Super Guru

@pankaj chaturvedi

inside your pig script do this:

set exectype=tez;

Re: Setting execution type as tez inside pig script.

New Contributor

As i mentioned in my question that i tried various variations of set exectype=tez like below but nothing worked

set exectype=tez

set exectype=TEZ

set ExecType=tez

exectype=tez

-x tez

Re: Setting execution type as tez inside pig script.

Mentor

Re: Setting execution type as tez inside pig script.

New Contributor

Semicolon at the end also does't work. I am currently using HDP sandbox 2.4 but still not able to set execution engine through script.

To Add I have also tried set exectype 'tez';set exectype='tez'; and other form.

Re: Setting execution type as tez inside pig script.

New Contributor

Yes that's right,i am also using sandbox 2.4 and its not working.

As per the documentation below it should work or may be documentation is wrong.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_dataintegration/content/ch_running-pig-t...

Is there any way to contact the Hortonwork team that prepared this document or any kind of support team that we can contact to raise this issue.

Re: Setting execution type as tez inside pig script.

Mentor

@pankaj chaturvedi @Sunile Manjee apparently the command is incorrect, typically set commands accepts key and value, there is no equal sign. So if this were to work it would be

SET exectype 'tez';

but it does not work in 2.4 or 2.5. I am following up internally whether this is a bug or feature.

Re: Setting execution type as tez inside pig script.

New Contributor

HDP 2.4 documentation (https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_dataintegration/content/ch_running-pig-tez.html) also says we can use "set exectype=tez;" with in a pig script. But its not working. Is this feature removed in the latest pig version?

Re: Setting execution type as tez inside pig script.

From the latest edition of the Pig book: However, it will not work if you put the statement set exectype 'tez' inside a Pig Latin script. This is because Pig decides the execution engine before parsing the script. It can be done by setting exectype in pig.properties, or using "-x tez" on the command line. "-x tez" will also work if your run Pig from Oozie-4.1+ (details here)

Re: Setting execution type as tez inside pig script.

Explorer

@pankaj chaturvedi You cannot make Pig run with the Tez execution engine via a Pig script. The "Pig Script" row in the first table of Running Pig with the Tez Execution Engine is wrong.

Instead, look at the same doc link and configure Pig to run with Tez by following the information in the "Command Line" or "Pig Properties" row of the table.

I will file a doc bug to have this fixed.

Let me know if this post does not solve your problem.

View solution in original post