question Re: Setting execution type as tez inside pig script. in Support Questions

Setting execution type as tez inside pig script.

pankaj311983 — Tue, 12 Jul 2016 10:19:44 GMT

Is there any way setting up execution mode as tez inside pig script.

i have tried various variation of set exectype=tez to use inside pig script but its not working.

I know it can be done through command line as pig -x tez -f filename

But in the HDPCD certification preparation task, question is asked like

Question: Write a pig script that satsifies following criteria

1. run the pig query using tez as execution engine

2. load data from hive table to pig using hcatalog

So i am confused whether we have to set something inside pig script that will make script run through TEZ.

Re: Setting execution type as tez inside pig script.

sunile_manjee — Tue, 12 Jul 2016 11:01:29 GMT

@pankaj chaturvedi

inside your pig script do this:

set exectype=tez;

Re: Setting execution type as tez inside pig script.

pankaj311983 — Tue, 12 Jul 2016 12:40:55 GMT

As i mentioned in my question that i tried various variations of set exectype=tez like below but nothing worked

set exectype=tez

set exectype=TEZ

set ExecType=tez

exectype=tez

-x tez

Re: Setting execution type as tez inside pig script.

aervits — Tue, 12 Jul 2016 15:54:59 GMT

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_dataintegration/content/ch_running-pig-tez.html

You forgot to put semicolon at the end.

SET exectype=tez;

Re: Setting execution type as tez inside pig script.

codecian — Fri, 15 Jul 2016 00:54:44 GMT

Semicolon at the end also does't work. I am currently using HDP sandbox 2.4 but still not able to set execution engine through script.

To Add I have also tried set exectype 'tez';set exectype='tez'; and other form.

Re: Setting execution type as tez inside pig script.

pankaj311983 — Fri, 15 Jul 2016 10:41:31 GMT

Yes that's right,i am also using sandbox 2.4 and its not working.

As per the documentation below it should work or may be documentation is wrong.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_dataintegration/content/ch_running-pig-tez.html

Is there any way to contact the Hortonwork team that prepared this document or any kind of support team that we can contact to raise this issue.

Re: Setting execution type as tez inside pig script.

ramu_valleti — Sat, 23 Jul 2016 14:57:01 GMT

HDP 2.4 documentation (https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_dataintegration/content/ch_running-pig-tez.html) also says we can use "set exectype=tez;" with in a pig script. But its not working. Is this feature removed in the latest pig version?

Re: Setting execution type as tez inside pig script.

pminovic — Tue, 26 Jul 2016 20:31:05 GMT

From the latest edition of the Pig book: However, it will not work if you put the statement set exectype 'tez' inside a Pig Latin script. This is because Pig decides the execution engine before parsing the script. It can be done by setting exectype in pig.properties, or using "-x tez" on the command line. "-x tez" will also work if your run Pig from Oozie-4.1+ (details here)

Re: Setting execution type as tez inside pig script.

aervits — Fri, 05 Aug 2016 02:04:12 GMT

@pankaj chaturvedi @Sunile Manjee apparently the command is incorrect, typically set commands accepts key and value, there is no equal sign. So if this were to work it would be

SET exectype 'tez';

but it does not work in 2.4 or 2.5. I am following up internally whether this is a bug or feature.

Re: Setting execution type as tez inside pig script.

fwelsch — Fri, 05 Aug 2016 03:54:58 GMT

@pankaj chaturvedi You cannot make Pig run with the Tez execution engine via a Pig script. The "Pig Script" row in the first table of Running Pig with the Tez Execution Engine is wrong.

Instead, look at the same doc link and configure Pig to run with Tez by following the information in the "Command Line" or "Pig Properties" row of the table.

I will file a doc bug to have this fixed.

Let me know if this post does not solve your problem.

Re: Setting execution type as tez inside pig script.

daijy — Fri, 05 Aug 2016 05:37:27 GMT

This is a mistake in document. You cannot "set exectype=tez;" in a pig script. You will need to set it in pig.properties, pig command line or JVM option.

Re: Setting execution type as tez inside pig script.

fwelsch — Fri, 05 Aug 2016 07:23:34 GMT

Also, you can use Java, as @daijy notes. So in summary you should be able to make Pig run on Tez by one of the methods in the first 3 rows of table 1 in Running Pig with the Tez Execution Engine (command line, the pig.properties file, or Java).

Re: Setting execution type as tez inside pig script.

pankaj311983 — Sun, 07 Aug 2016 10:04:32 GMT

Thanks for the clarification, if possible let us know once documentation is corrected.