I have recently installed cloudera quick start vm docker image on mac m1 , have noticed in this 6 years old image ican't find kudu /kafka / nifi , below are the details
Hadoop 2.6.0-cdh5.7.0 Subversion http://github.com/cloudera/hadoop -r c00978c67b0d3fe9f3b896b5030741bd40bf541a Compiled by jenkins on 2016-03-23T18:36Z Compiled with protoc 2.5.0 From source with checksum b2eabfa328e763c88cb14168f9b372 This command was run using /usr/jars/hadoop-common-2.6.0-cdh5.7.0.jar
how to install additional applications from already pulled docker image from docker hub is it possible to install additional missed applications in the same docker ? ,if the answer is yes how and leads ?
@BigDataAvengers can you tell us a bit about why you're installing this six year old image? Does what you've already installed work with acceptable responsiveness? You didn't say:
where you retrieved the docker quickstart image from
how much RAM your mac m1 has available
whether or not you were actually able to successfully start up the services, beginning with the Cloudera Manager user interface, that were already present in the cloudera quickstart vm image you installed
The docker image you've installed appears to be based on CDH 5.7, which went out of support at the end of August in 2019 and is even older than the most recent version of the CDH Quickstart, which was based on CDH 5.13, which itself is already out of support. Cloudera Enterprise 5.13 reached it's end of support date in October 2020 (open that link and then expand the section labeled "Cloudera Enterprise products" underneath Current End of Support (EoS) Dates). For this reason, Cloudera is no longer distributing the CDH 5-based Quickstart. It is also why this hyperlink:
Next, it's not at all certain that your M1 Mac has enough RAM available in total to be able to run the services you are attempting to run within the container (or containers) without consuming so much of the host machine's memory that OS X starts killing processes to free up memory, and Docker is very much one of the processes subject to being killed. IIRC the memory recommendation for the Docker version of the Quickstart VM itself was about 10 GB, so on a 16GB Mac, with the overhead for Docker and other running applications (such as a web browser), things will be running pretty tight as far as memory goes. Even if you were able to install the docker image that @ckumar pointed you to in another container alongside the quickstart's container, that does not mean that it would provide a useable system, because the HDF NiFi container called for at least 8 GB.
So while it is possible that someone with the requisite knowledge of the Linux command line and Docker combined with sufficient skill and abilities with the various required development tools could update the last, outdated cloudera quickstart image to add Kudu and then subsequently figure out how to run it alongside containers for some version of NiFi and Kafka (and indeed, some member of the Cloudera Community may have already done so and be willing to share their method in response to your question), it doesn't mean you'd have the necessary hardware resources to run it on your M1 Mac, or that it would produce acceptable results.
You should ask yourself whether or not this would be worthwhile.
The current Enterprise Data Platform offered by Cloudera as of this writing is Cloudera Data Platform (CDP), which in it's on-premises "form factor" is now called CDP Private Cloud. CDP supersedes CDH and the components are at the appropriate versions that enable them to work together.
If you're just looking to evaluate a data platform, you can currently do so without a Cloudera subscription by downloading and installing the Trial Version of CDP Private Cloud Base Edition of Cloudera Data Platform. Cloudera has an extensive tutorial on installing CDP Private Cloud Base in an Infrastructure as a service (IaaS) fashion using AWS on its website. This approach allows you to leverage machines which have hardware resources, such as plentiful RAM, that your laptop might not have available. And CDP Private Cloud Base Edition ships with Kudu.
Bill Brooks, Community Moderator Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.