Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Setting execution type as tez inside pig script.

avatar

Is there any way setting up execution mode as tez inside pig script.

i have tried various variation of set exectype=tez to use inside pig script but its not working.

I know it can be done through command line as pig -x tez -f filename

But in the HDPCD certification preparation task, question is asked like

Question: Write a pig script that satsifies following criteria

1. run the pig query using tez as execution engine

2. load data from hive table to pig using hcatalog

So i am confused whether we have to set something inside pig script that will make script run through TEZ.

1 ACCEPTED SOLUTION

avatar
Rising Star

@pankaj chaturvedi You cannot make Pig run with the Tez execution engine via a Pig script. The "Pig Script" row in the first table of Running Pig with the Tez Execution Engine is wrong.

Instead, look at the same doc link and configure Pig to run with Tez by following the information in the "Command Line" or "Pig Properties" row of the table.

I will file a doc bug to have this fixed.

Let me know if this post does not solve your problem.

View solution in original post

12 REPLIES 12

avatar
Master Guru

@pankaj chaturvedi

inside your pig script do this:

set exectype=tez;

avatar

As i mentioned in my question that i tried various variations of set exectype=tez like below but nothing worked

set exectype=tez

set exectype=TEZ

set ExecType=tez

exectype=tez

-x tez

avatar
Master Mentor

avatar
New Contributor

Semicolon at the end also does't work. I am currently using HDP sandbox 2.4 but still not able to set execution engine through script.

To Add I have also tried set exectype 'tez';set exectype='tez'; and other form.

avatar

Yes that's right,i am also using sandbox 2.4 and its not working.

As per the documentation below it should work or may be documentation is wrong.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_dataintegration/content/ch_running-pig-t...

Is there any way to contact the Hortonwork team that prepared this document or any kind of support team that we can contact to raise this issue.

avatar
Master Mentor

@pankaj chaturvedi @Sunile Manjee apparently the command is incorrect, typically set commands accepts key and value, there is no equal sign. So if this were to work it would be

SET exectype 'tez';

but it does not work in 2.4 or 2.5. I am following up internally whether this is a bug or feature.

avatar

HDP 2.4 documentation (https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_dataintegration/content/ch_running-pig-tez.html) also says we can use "set exectype=tez;" with in a pig script. But its not working. Is this feature removed in the latest pig version?

avatar
Master Guru

From the latest edition of the Pig book: However, it will not work if you put the statement set exectype 'tez' inside a Pig Latin script. This is because Pig decides the execution engine before parsing the script. It can be done by setting exectype in pig.properties, or using "-x tez" on the command line. "-x tez" will also work if your run Pig from Oozie-4.1+ (details here)

avatar
Rising Star

@pankaj chaturvedi You cannot make Pig run with the Tez execution engine via a Pig script. The "Pig Script" row in the first table of Running Pig with the Tez Execution Engine is wrong.

Instead, look at the same doc link and configure Pig to run with Tez by following the information in the "Command Line" or "Pig Properties" row of the table.

I will file a doc bug to have this fixed.

Let me know if this post does not solve your problem.