Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDFS performance issue with HDP-2.6

avatar
Explorer

We have faced a HDFS performance issue with HDP-2.6. When i/o intensive jobs are submitted, many BlockSender threads are waited and the number of ThreadsBlocked raises. Then, the performance is down.

We investigated the cause and found that the bug is generated by HDFS-11160 and it fixed by HDFS-11187 which isn't included in the latest HDP-2.6.

Is there any plan to release a new HDP-2.6? If there is the plan, I hope it includes HDFS-11187.

(BTW, HDP-3.0 won't occur the issue since it includes HDFS-11187.)

1 ACCEPTED SOLUTION

avatar

Hi @Takanobu Asanuma ,

Happened to see in Hadoop-HDFS code repo. the issue you are referring to is Fixed in HDP-3.0.1. so I assume HDP-3.0 even don't have the fix.

also cant find it in fixed issues in hortonworks docs : https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/release-notes/content/fixed_issues.html

i would suggest you go ahead and create a Backport Request in hortonworks support portal which will be analyzed by engineers and provided a hotfix if the backport ing is possible?

seeing the code I think backporting won't be an issue.

Please login and accept this answer if you find this helpful 🙂

View solution in original post

4 REPLIES 4

avatar
Expert Contributor

Is there any plan to release a new HDP-2.6?

-No, I think.

avatar
Explorer

Thanks for your reply, @schhabra.

avatar

Hi @Takanobu Asanuma ,

Happened to see in Hadoop-HDFS code repo. the issue you are referring to is Fixed in HDP-3.0.1. so I assume HDP-3.0 even don't have the fix.

also cant find it in fixed issues in hortonworks docs : https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/release-notes/content/fixed_issues.html

i would suggest you go ahead and create a Backport Request in hortonworks support portal which will be analyzed by engineers and provided a hotfix if the backport ing is possible?

seeing the code I think backporting won't be an issue.

Please login and accept this answer if you find this helpful 🙂

avatar
Explorer

Thanks for your comments, @Akhil S Naik.

IIUC, HDP-3.0 is HDP-3.0.0.0-1634 which includes the fix.
https://github.com/hortonworks/hadoop-release/commit/2021f4bdce3b27c46edaad198f0007a26a8a1391

We will consider the backport request.

I'd like to accept your answer. Thanks again.