Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Default execution engine for Pig changed in HDP 2.6.1.0 ?

avatar
Contributor

Hello HCC,

We recently upgraded our cluster from HDP 2.5.3.0 to HDP 2.6.1.0 and Ambari from 2.4.2 to 2.5.2 and we observed in pig version changes after HDP upgrade pig default execution engine changed from MR mode to TEZ execution engine, is it a bug or expected behavior, i gone through HDP 2.6.1.0 release notes but no where its mentioned about this change.

Can someone please clarify about this change 🙂

Thanks,

Jalender.

@Orlando Teixeira @Kuldeep Kulkarni @kkanchu


change.png
1 ACCEPTED SOLUTION

avatar
Master Mentor
5 REPLIES 5

avatar
Master Mentor

@Jalender Reddy

I see the changes happened from : Switch Pig default engine to tez in HDP 2.6: https://issues.apache.org/jira/browse/AMBARI-19702

The Git commit reference: https://github.com/apache/ambari/commit/2f06c1cefac30e0a53d631cf2a2a6aef64a89e0c

.

avatar
Master Mentor

@Jalender Reddy

It seem to have happened as part of [BUG-73126] which is also related to [AMBARI-19702]


And the Fix Version/s are : HDP-2.6.0.0 , Ambari-2.5.0

As the [BUG-73126]* link is private hence if you want more details about it then please contact Hortonworks Support.

.

avatar
Master Mentor

@Jalender Reddy

A reference of BUG-73126 is also mentioned in Ambari Doc:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.0.3/bk_ambari-release-notes/content/ambari_reln...

.

Also if you will check the commit for [AMBARI-19702] then you will find that the changes are also made to the HDP upgrade files as well.

https://github.com/apache/ambari/commit/772c093073b31fb0887d7072622e4c37c227e28d

So during HDP upgrade via ambari the Engine will be changed to Tez.

avatar
Contributor

Hi @Jay SenSharma

We are already on HDP 2.6.1.0 and Ambari 2.5.2.0, as you mentioned it was fixed in "And the Fix Version/s are : HDP-2.6.0.0 , Ambari-2.5.0" not sure why we hit the issue on HDP 2.6.1

Thanks,

Jalender.

avatar
Master Mentor

@Jalender Reddy

The Fix is to Have the TEZ as default engine from now on.