Hi @Zinway  Assuming that you have a valid Cloudera subscription, you should be able to log into your My Cloudera account and navigate to the support portal in order to get in contact with Support and initiate a request for that specific .tar.gz file.     ... View more

@Vijay11  I cannot offer you any assurances that Cloudera maintains a repository for Ambari 2.7.6. I say that because, while Cloudera continues to support customers using Ambari in accordance with its associated products' support lifecycle, Cloudera announced it was ending its involvement in the Ambari open source community in 2021. The part of that announcement that is relevant to your questions is this paragraph: Please note that Cloudera does not support direct installation of Apache Ambari 2.7.6. Although 2.7.6 contains numerous patches provided by Cloudera (some of which were delivered separately as hotfix releases to customers); the 2.7.6 release is not itself a Cloudera release. You should carefully read the whole thing.   As far as how you can get paid access to access the private repositories and the costs for a Cloudera subscription, the only way to obtain answers to those questions is to get in touch with the Cloudera sales team. Once you make contact with someone there, be sure to tell them that you're looking to gain access to a repository, if any, for Ambari 2.7.6.. Note that as of this writing, Cloudera recommends upgrading to the latest version of CDP Private Cloud Base as soon as possible, as it has a richer set of features, is up-to-date and remains under standard Cloudera support through August 2024.     ... View more

Hi @DzBoris  May I ask why you are attempting to download CDH 5.11.1? And does your organization have a valid Cloudera subscription?   Cloudera Enterprise 5.11 became generally available in June of 2017. As you no doubt are aware, that was quite a while ago, especially in terms of "internet time". Cloudera Enterprise 5.11 reached it's end of support date in April 2020 (open that link and then expand the section labeled "Cloudera Enterprise products" underneath Current End of Support (EoS) Dates).   The current Enterprise Data Platform offered by Cloudera is Cloudera Data Platform (CDP), which in it's on-premises "form factor" is offered as CDP Private Cloud. CDP supersedes CDH as it is fairly up to date on all the included components, which is not the case with CDH 5.11.   As a general matter, the credentials to access the private repository where Cloudera is now distributing previous versions of CDH are not generally the same ones to access Cloudera's website or the Cloudera community. Employees of organizations with a valid Cloudera subscription can upgrade their versions of Cloudera Manager to a newer version that uses the modified URLs which can contain these credentials. You can get started reading about how to do this in the Cloudera Enterprise 5.x Release Notes here: Version, Packaging, and Download Information.   The use of credentials which are not tied to a valid Cloudera subscription is the most common cause of the HTTP 403 Forbidden error message you are encountering.     ... View more

@joshtheflame  I just wanted to provide a bit more context. The partial page shot you've included above appears to show Cloudera Manager running against a CDH 6.1.0 cluster. CDH 6.1.0 was released in December of 2018. As you no doubt are aware, that was quite a while ago, especially in terms of "internet time". Hopefully you are aware that CDH 6.1.x has reached its End of Support (EoS). You can find the most recent official reminder of the previous announcement that Cloudera Enterprise 6.x reached End of Support (EoS) in 2021 here: March 2021 Customer Advisory - 2: End of Support for Cloudera Products (CDH/CM 6.x & HDP 3.x). Cloudera's lifecycle support policies are documented here: https://www.cloudera.com/legal/policies/support-lifecycle-policy.html My understanding is that organizations with a valid Cloudera subscription for legacy products such as CDH would have been sent this announcement directly.   If that screenshot represents what your bank is running in production, I would recommend that you reach out to your Cloudera Account team and discuss your upgrade options as soon as possible. You are going to have to upgrade in order to take advantage of any of the offerings mentioned in @steven-matison 's reply earlier.     ... View more

I just want to address the assertions in the first paragraph above.   I don't think it's quite correct to deem HDF "a frozen project". it would be more accurate to say that, as a Cloudera product, it was superseded by Cloudera DataFlow (CDF) in the Winter of 2019. The legacy product HDF has already reached it's End of Support (EoS) date in March 2022, so it is safe to assume that no new major updates with new versions of NiFi will be going into it.   There'a always some differences between the versions released by Cloudera in products such as CDF and the release of "upstream" component projects such as Apache NiFi. This is analogous to how there are differences between what mainline kernel is "current" in the open source Linux world and what RedHat, for example, releases as part of Red Hat Enterprise Linux.   Right now, Apache NiFi is a component in a product called Cloudera Flow Management (CFM). My understanding is that CFM 2.1.4 will be based on NiFi 1.16 when it becomes available a little later this year. To find out more about that release schedule, reach out to your Cloudera Account Team. I haven't personally tried doing it, but the documentation for CFM 2.1.1, the currently-released version, indicates that you can install CFM on top of HDF Version 3.5.2, so when the new version comes out, you might not be out of luck if you want to run NiFi version 1.16.x on top of that version of HDF, if CFM maintains that support (I can't offer any assurances that it will, however, given that HDF has reached EoS already).   Now that brings us to Ambari. While Cloudera continues to support customers using Ambari in accordance with its associated products' support lifecycle, including HDF, Cloudera announced it was ending its involvement in the Ambari open source community in 2021. You are correct that at the ASF, Ambari was terminated in January of this year. That means that going forward, there are going to be fewer people using it, and fewer people who have the knowledge of how it works in order to effectively support it. This is one reason that as of this writing, Cloudera recommends upgrading to the latest version of CDP Private Cloud Base as soon as possible, as it has a richer set of features and is the successor to HDF and remains under standard Cloudera support through August 2024.   We welcome your questions, and this thread will remain visible here in the hope that some other member of the Cloudera Community will reply with assistance addressing your second paragraph above.     ... View more

Hi @sparkdeveloper  You are encountering the well-known phenomenon of Java Class shadowing, and it really doesn't have all that much to do with Spark, or the spark-submit utility. One direct solution is to employ a technique called "shading" the dependencies, which usually winds up being a lot of work, and I personally avoid doing it. If you want to go that route, you can read all about that technique on any number of websites that cover Java software development or in books on the same topic.   Obviously, I don't know what the code you're running that uses this library does, but if you're looking for some way to resolve this quickly, I think you are better off rewriting your code that uses the 3.2.0 version of this library to use the APIs in the version that comes pre-packaged with your spark installation and just omit adding your .jar file to the CLASSPATH. You'll probably need to download the HikariCP-2.5.1.jar file and install it where you're doing your development work. I realize that is a lazy kind of answer, but you should consider it unless you really have a compelling need to use the later version.     ... View more

@aval  I don't personally have the required expertise (yet) to answer your question, but I did want to attempt to clarify the question for other community members who do.   In paragraph 2, you write: We want to setup the data experiences so we can move some work between the cdp public cloud and the cdp private cloud experiences since the control planes are similar. I think, based on the rest of your question, that you intended to write "…so we can move some work between cdp private cloud base and the cdp private cloud experiences since the control planes are similar.". It's important to be clear about this because Cloudera does have a product called CDP public cloud, but that "form factor" of CDP only works on the infrastructure provided by the so-called hyperscalers, or CSPs such as AWS, Azure and GCP. The on-premises offering is called CDP Private Cloud. Please update this thread with a new, clarifying post if you really want to know how to move workloads between cdp public cloud and cdp private cloud experiences clusters.     ... View more

Hi @Freschone  May I ask why you need to download Quickstart VM based on CDH 5.10? Is this a classroom assignment?   As a general matter, Cloudera is no longer updating or making the Cloudera Quickstart VM available for download (and hasn't since March of 2020) because it was outdated and obsolete as the last version was based on CDH 5.13, which went out of support in the Fall of 2020.   The credentials to access the private repository where Cloudera is now distributing previous versions of CDH are not are not generally the same ones to access Cloudera's website or the Cloudera community. Employees of organizations with a valid Cloudera subscription can generate repository credentials from a CDH license key, and there is a full description of how to do this in the Cloudera Enterprise 6.x Release Notes here: Version, Packaging, and Download Information.     ... View more

Hi @Yaniv  The link to the GiHub repo that @GangWar shared earlier in the thread is the canonical location for assets related to Cloudera's Ansible-based automation for the deployment of CDP Private Cloud Base. As it says in the blog post referred to in this thread earlier: The Ansible playbooks are provided on an as-is basis without any warranty or support. If you can't find a desired asset there, that means it has not been publicly released by Cloudera. To put it more directly, as of April 2022, Cloudera has not yet publicly published a definition that includes HA.   If you go ahead and develop an HA template on your own, you are always welcome to raise an issue on the GitHub project and submit your in-house developed template as a pull request, as Cloudera welcomes participation from the community.   If you don't have the Ansible playbook development skills or the time to obtain them, I would suggest you engage Professional Services to develop the assets you need. If you're a Cloudera Subscription Support customer, please do reach out to your Account team to discuss your potential project if you wish to go that route.     ... View more

Hi @azg ,   Do you know what version of NiFi is included in the docker image you're running? And when you say "the connection works well on localhost", can you expand on what exactly you did to confirm that the connection "works"?     ... View more