question Re: Does pig uses mapreduce in backend in tezmode? in Archives of Support Questions (Read Only)

Does pig uses mapreduce in backend in tezmode?

mhdeshmukh22 — Mon, 29 Feb 2016 02:44:17 GMT

Re: Does pig uses mapreduce in backend in tezmode?

bleonhardi — Mon, 29 Feb 2016 03:09:51 GMT

Its configurable, run pig -x tez and it will use tez.

Tez is a processing engine not too unlikely to spark that can run mapreduce tasks but many other tasks as well. For Pig tez executes the mapreduce code that is generated by pig but it handles the shuffle etc. in its own way.

Re: Does pig uses mapreduce in backend in tezmode?

aervits — Mon, 29 Feb 2016 03:59:01 GMT

There are four modes available, mapreduce, local, tez and tez_local. You pass the mode with -x switch. With version 0.15 of pig, tez and tez_local are recommended for better performance, in tez modes pig uses tez execution engine and by default it is still mapreduce mode, in which pig uses MR as execution engine. At some point tez will be a default engine.

Re: Does pig uses mapreduce in backend in tezmode?

nsabharwal — Mon, 29 Feb 2016 09:04:29 GMT

@Mahesh Deshmukh

Short answer is Yes and you can configure pig as you like

Please see this tutorial http://hortonworks.com/hadoop-tutorial/faster-pig-tez/

Re: Does pig uses mapreduce in backend in tezmode?

mhdeshmukh22 — Wed, 02 Mar 2016 12:28:31 GMT

thanks all for your response.

Re: Does pig uses mapreduce in backend in tezmode?

nsabharwal — Wed, 02 Mar 2016 13:19:58 GMT

@Mahesh Deshmukh You can close the thread as looks like all the answers are almost same.