Skip to main content
MidoMido
MidoMido
Silver ⭐️
July 20, 2020
Solved

Hadoop 3.1.2 with CDH

I will deploy CDH 6.3.3 and it has Hadoop 3.0.0 , Is there any way to use later version of Hadoop like 3.1.2 as more stable instead and still being supported with Cloudera CDH 6.3.3 ?

Tim Armstrong
Tim ArmstrongAnswer
Platinum ⭐️
July 20, 2020

I think you're misunderstanding what CDH is. Hadoop in CDH is not a straight repackaging of an upstream Apache Hadoop release - it is based on an Apache Hadoop release but with a lot of enhancements, security and bug fixes based on our own testing and integration work and our experience working with customers running this in production.

 

Our goal is that it should be more production-ready and battle tested than any Apache Hadoop release.

 

So CDH 6.3.3 includes a lot of the improvements from post-3.0.0 Hadoop versions. If you want to see what was added in each version, the release notes have a lot of info - https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_6_release_notes.html#cdh6_release_notes

View in thread

3 replies

Tim Armstrong
Platinum ⭐️
Tim ArmstrongAnswer
Platinum ⭐️
July 20, 2020

I think you're misunderstanding what CDH is. Hadoop in CDH is not a straight repackaging of an upstream Apache Hadoop release - it is based on an Apache Hadoop release but with a lot of enhancements, security and bug fixes based on our own testing and integration work and our experience working with customers running this in production.

 

Our goal is that it should be more production-ready and battle tested than any Apache Hadoop release.

 

So CDH 6.3.3 includes a lot of the improvements from post-3.0.0 Hadoop versions. If you want to see what was added in each version, the release notes have a lot of info - https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_6_release_notes.html#cdh6_release_notes

Translate
MidoMido
MidoMidoAuthor
Silver ⭐️
July 20, 2020

Thanks Tim for the clarification..however owner still wants to make benefits of Hadoop later version..is there any work around for this or no way ?

Translate
Tim Armstrong
Platinum ⭐️
Tim Armstrong
Platinum ⭐️
July 21, 2020

I really would suggest looking at whether the particular feature you want are in CDH6.3.3. We do backport a lot of features. E.g the GPU scheduling features for YARN for Hadoop 3.1 were included in CDH 6.2 https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_620_new_features.html#hadoop_new_620.

 

If the question is whether you can run a non-CDH version of Hadoop, and still be running CDH, then the answer is no. Or if non-CDH releases of Hadoop are supported by Cloudera - also no. We only release and support CDH versions that have been fully integrated and tested against the other CDH components.

 

If the question is whether there is a way to take Apache Hadoop release and deploy it in a Cloudera Manager cluster, then no - it's not packaged in the right way

Translate
Terms & ConditionsAccessibility statement