question Re: Default execution engine for Pig changed in HDP 2.6.1.0 ? in Archives of Support Questions (Read Only)

Default execution engine for Pig changed in HDP 2.6.1.0 ?

reddyr211 — Thu, 26 Oct 2017 08:28:28 GMT

Hello HCC,

We recently upgraded our cluster from HDP 2.5.3.0 to HDP 2.6.1.0 and Ambari from 2.4.2 to 2.5.2 and we observed in pig version changes after HDP upgrade pig default execution engine changed from MR mode to TEZ execution engine, is it a bug or expected behavior, i gone through HDP 2.6.1.0 release notes but no where its mentioned about this change.

Can someone please clarify about this change 🙂

Thanks,

Jalender.

@Orlando Teixeira @Kuldeep Kulkarni @kkanchu

Re: Default execution engine for Pig changed in HDP 2.6.1.0 ?

jsensharma — Thu, 26 Oct 2017 09:21:02 GMT

@Jalender Reddy

I see the changes happened from : Switch Pig default engine to tez in HDP 2.6: https://issues.apache.org/jira/browse/AMBARI-19702

The Git commit reference: https://github.com/apache/ambari/commit/2f06c1cefac30e0a53d631cf2a2a6aef64a89e0c

Re: Default execution engine for Pig changed in HDP 2.6.1.0 ?

jsensharma — Thu, 26 Oct 2017 23:49:45 GMT

@Jalender Reddy

It seem to have happened as part of [BUG-73126] which is also related to [AMBARI-19702]

And the Fix Version/s are : HDP-2.6.0.0 , Ambari-2.5.0

As the [BUG-73126]* link is private hence if you want more details about it then please contact Hortonworks Support.

Re: Default execution engine for Pig changed in HDP 2.6.1.0 ?

jsensharma — Thu, 26 Oct 2017 23:50:41 GMT

@Jalender Reddy

A reference of BUG-73126 is also mentioned in Ambari Doc:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.0.3/bk_ambari-release-notes/content/ambari_relnotes-2.5.0.3-fixed-issues.html

Also if you will check the commit for [AMBARI-19702] then you will find that the changes are also made to the HDP upgrade files as well.

https://github.com/apache/ambari/commit/772c093073b31fb0887d7072622e4c37c227e28d

So during HDP upgrade via ambari the Engine will be changed to Tez.

Re: Default execution engine for Pig changed in HDP 2.6.1.0 ?

reddyr211 — Sat, 28 Oct 2017 01:25:44 GMT

Hi @Jay SenSharma

We are already on HDP 2.6.1.0 and Ambari 2.5.2.0, as you mentioned it was fixed in "And the Fix Version/s are : HDP-2.6.0.0 , Ambari-2.5.0" not sure why we hit the issue on HDP 2.6.1

Thanks,

Jalender.

Re: Default execution engine for Pig changed in HDP 2.6.1.0 ?

jsensharma — Sat, 28 Oct 2017 01:31:37 GMT

@Jalender Reddy

The Fix is to Have the TEZ as default engine from now on.