We are trying to optimize cluster provisioning by preloading parcels into our VM images. The goal is to have parcels already available on each host before they are added to the cluster, so we can avoid re-distribution during host onboarding. Here is what we tested and observed: We baked parcels into the VM image under /opt/cloudera/parcels/. When we add a host created from this image into the cluster, Cloudera Manager still triggers parcel distribution. During this process, CM deletes the preloaded parcels on the new host and redistributes them again. We have tested the following without success: Setting Auto Distribution = false Verifying parcel existence manually Creating/updating the .distributed_parcels file with the correct versions Our conclusion so far is that Cloudera Manager does not trust preloaded parcels, but instead enforces consistency against its internal parcel state database, which causes redistribution and deletion even if parcels are already present. Our question: Is there any supported way to have parcels preloaded on all hosts (via a baked image or other method) so that Cloudera Manager recognizes them as already distributed? If not, is the recommended approach instead?
... View more
