Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS performance issue with HDP-2.6

Solved Go to solution

HDFS performance issue with HDP-2.6

New Contributor

We have faced a HDFS performance issue with HDP-2.6. When i/o intensive jobs are submitted, many BlockSender threads are waited and the number of ThreadsBlocked raises. Then, the performance is down.

We investigated the cause and found that the bug is generated by HDFS-11160 and it fixed by HDFS-11187 which isn't included in the latest HDP-2.6.

Is there any plan to release a new HDP-2.6? If there is the plan, I hope it includes HDFS-11187.

(BTW, HDP-3.0 won't occur the issue since it includes HDFS-11187.)

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HDFS performance issue with HDP-2.6

Hi @Takanobu Asanuma ,

Happened to see in Hadoop-HDFS code repo. the issue you are referring to is Fixed in HDP-3.0.1. so I assume HDP-3.0 even don't have the fix.

also cant find it in fixed issues in hortonworks docs : https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/release-notes/content/fixed_issues.html

i would suggest you go ahead and create a Backport Request in hortonworks support portal which will be analyzed by engineers and provided a hotfix if the backport ing is possible?

seeing the code I think backporting won't be an issue.

Please login and accept this answer if you find this helpful :)

4 REPLIES 4

Re: HDFS performance issue with HDP-2.6

Expert Contributor

Is there any plan to release a new HDP-2.6?

-No, I think.

Re: HDFS performance issue with HDP-2.6

New Contributor

Thanks for your reply, @schhabra.

Re: HDFS performance issue with HDP-2.6

Hi @Takanobu Asanuma ,

Happened to see in Hadoop-HDFS code repo. the issue you are referring to is Fixed in HDP-3.0.1. so I assume HDP-3.0 even don't have the fix.

also cant find it in fixed issues in hortonworks docs : https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/release-notes/content/fixed_issues.html

i would suggest you go ahead and create a Backport Request in hortonworks support portal which will be analyzed by engineers and provided a hotfix if the backport ing is possible?

seeing the code I think backporting won't be an issue.

Please login and accept this answer if you find this helpful :)

Re: HDFS performance issue with HDP-2.6

New Contributor

Thanks for your comments, @Akhil S Naik.

IIUC, HDP-3.0 is HDP-3.0.0.0-1634 which includes the fix.
https://github.com/hortonworks/hadoop-release/commit/2021f4bdce3b27c46edaad198f0007a26a8a1391

We will consider the backport request.

I'd like to accept your answer. Thanks again.