Default execution engine for Pig changed in HDP 2.6.1.0 ?

[#1]

Hello HCC,

We recently upgraded our cluster from HDP 2.5.3.0 to HDP 2.6.1.0 and Ambari from 2.4.2 to 2.5.2 and we observed in pig version changes after HDP upgrade pig default execution engine changed from MR mode to TEZ execution engine, is it a bug or expected behavior, i gone through HDP 2.6.1.0 release notes but no where its mentioned about this change.

Can someone please clarify about this change 🙂

Thanks,

Jalender.

@Orlando Teixeira @Kuldeep Kulkarni @kkanchu

[#2 · Accepted solution]

@Jalender Reddy

I see the changes happened from : Switch Pig default engine to tez in HDP 2.6: https://issues.apache.org/jira/browse/AMBARI-19702

The Git commit reference: https://github.com/apache/ambari/commit/2f06c1cefac30e0a53d631cf2a2a6aef64a89e0c

.

[#3]

@Jalender Reddy

It seem to have happened as part of [BUG-73126] which is also related to [AMBARI-19702]

And the Fix Version/s are : HDP-2.6.0.0 , Ambari-2.5.0

As the [BUG-73126]* link is private hence if you want more details about it then please contact Hortonworks Support.

.

[#4]

@Jalender Reddy

A reference of BUG-73126 is also mentioned in Ambari Doc:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.0.3/bk_ambari-release-notes/content/ambari_reln...

.

Also if you will check the commit for [AMBARI-19702] then you will find that the changes are also made to the HDP upgrade files as well.

https://github.com/apache/ambari/commit/772c093073b31fb0887d7072622e4c37c227e28d

So during HDP upgrade via ambari the Engine will be changed to Tez.

[#5]

Hi @Jay SenSharma

We are already on HDP 2.6.1.0 and Ambari 2.5.2.0, as you mentioned it was fixed in "And the Fix Version/s are : HDP-2.6.0.0 , Ambari-2.5.0" not sure why we hit the issue on HDP 2.6.1

Thanks,

Jalender.

[#6]

@Jalender Reddy

The Fix is to Have the TEZ as default engine from now on.