question Re: Hadoop 3.1.2 with CDH in Support Questions

Hadoop 3.1.2 with CDH

MidoMido — Fri, 16 Sep 2022 14:38:24 GMT

I will deploy CDH 6.3.3 and it has Hadoop 3.0.0 , Is there any way to use later version of Hadoop like 3.1.2 as more stable instead and still being supported with Cloudera CDH 6.3.3 ?

Re: Hadoop 3.1.2 with CDH

Tim Armstrong — Mon, 20 Jul 2020 19:33:00 GMT

I think you're misunderstanding what CDH is. Hadoop in CDH is not a straight repackaging of an upstream Apache Hadoop release - it is based on an Apache Hadoop release but with a lot of enhancements, security and bug fixes based on our own testing and integration work and our experience working with customers running this in production.

Our goal is that it should be more production-ready and battle tested than any Apache Hadoop release.

So CDH 6.3.3 includes a lot of the improvements from post-3.0.0 Hadoop versions. If you want to see what was added in each version, the release notes have a lot of info - https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_6_release_notes.html#cdh6_release_notes

Re: Hadoop 3.1.2 with CDH

MidoMido — Mon, 20 Jul 2020 22:19:24 GMT

Thanks Tim for the clarification..however owner still wants to make benefits of Hadoop later version..is there any work around for this or no way ?

Re: Hadoop 3.1.2 with CDH

Tim Armstrong — Tue, 21 Jul 2020 00:49:55 GMT

I really would suggest looking at whether the particular feature you want are in CDH6.3.3. We do backport a lot of features. E.g the GPU scheduling features for YARN for Hadoop 3.1 were included in CDH 6.2 https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_620_new_features.html#hadoop_new_620.

If the question is whether you can run a non-CDH version of Hadoop, and still be running CDH, then the answer is no. Or if non-CDH releases of Hadoop are supported by Cloudera - also no. We only release and support CDH versions that have been fully integrated and tested against the other CDH components.

If the question is whether there is a way to take Apache Hadoop release and deploy it in a Cloudera Manager cluster, then no - it's not packaged in the right way