Hi All,   I am trying all means to find out a latest Cloudera sandbox for practicing purpose on mac m1 , yes i know and experienced many issues due to M1 not supporting most of the softwares with out any tweaks.   I have found few options   Option 1 - Virtual box -> not compatible for M1 Option 2 - Parallels -> not open source  Option 3 - VM ware fusion  -> not open source  Option 4 - UTM - again some issues keyboard no sync  Option 5 - Docker desktop -> This options works with tweaks   I have picked up Docker desktop ,since there is a software available for M1 , i have then explored the options  to find a suitable cloudera/hortonwork sandbox images   1. Docker - Hortonworks sandbox 3.0.1 -> failed Hortonworks 3.0.1 -> I have tried with some tweaks but not able to configure in spite of spending a lot of time fixing the issues ,no luck    Issue/error - Failed to get D-Bus connection: No such file or directory my explanation about the error including the log https://stackoverflow.com/questions/71489739/deploying-hortonworks-sandbox-on-docker-on-mac-m1-installation-error-arm64/71521909?noredirect=1#comment126482374_71521909   2. Docker - Cloudera sandbox CDH 6.3.0 --> failed       docker run -m 8G --hostname=cloudera.sandbox-cdh:6.3.0 --privileged=true -t -i -v /Users/appleworld/Documents/Docker:/Src --publish-all=true -p 8888 -p 7180:7180 39f9547152c6 /usr/sbin/int docker-clouderasandbox-cdh:6.3.0 --platform amd64       the above docker run command i suspect is incorrect ? , ccan you please correct    issue/errors     jps -> no service information available Failed to mount tmpfs at /run: Operation not permitted Failed to mount cgroup at /sys/fs/cgroup/systemd: Operation not permitted [!!!!!!] Failed to mount API filesystems, freezing   Docker hub Cloudera sandbox image  https://hub.docker.com/r/cloudera/sandbox-cdh/tags?page=1&ordering=-last_updated   3. Docker - Cloudera quickstart vm docker image --> works fine on mac m1   https://hub.docker.com/r/cloudera/quickstart     docker run -m 8G --hostname=quickstart.cloudera --privileged=true -t -i -v /Users/appleworld/Documents/Docker:/Src --publish-all=true -p 8888 -p 7180:7180 cloudera/quickstart /usr/bin/docker-quickstart --platform amd64     # I have also prepared step by step installation https://medium.com/@rakeshgopidi/installing-cloudera-quickstart-vm-through-docker-hub-on-mac-m1-879f4a3d0fd4   I really want to know why 1. Docker - Hortonworks sandbox 3.0.1 -> failed & 2. Docker - Cloudera sandbox CDH 6.3.0 is not working or not able to fix    can ideas to fix ?  ... View more

I am trying to setup "Deploying Hortonworks Sandbox on Docker" on MAC M1 for practice , unfortunately i am not able to find a solution in spite of google search /stock overflow Official Cloudera Installation directory https://www.cloudera.com/tutorials/sandbox-deployment-and-install-guide/3.html Reference installation https://javamana.com/2021/02/20210206191356735v.html   below is my configuration Ram - 16gb - i have allocated 12 gb Rosetta2 Installed Docker installed Mac M1   Docker hub image link https://hub.docker.com/r/hortonworks/sandbox-hdp/tags docker pull hortonworks/sandbox-hdp:3.0.1 --platform linux/amd64   Script       #!/usr/bin/env sh #This script downloads HDP sandbox along with their proxy docker container set -x # CAN EDIT THESE VALUES registry="hortonworks" name="sandbox-hdp" version="3.0.1" proxyName="sandbox-proxy" proxyVersion="1.0" flavor="hdp" # NO EDITS BEYOND THIS LINE # housekeeping echo $flavor > sandbox-flavor # create necessary folders for nginx and copy over our rule generation script there mkdir -p sandbox/proxy/conf.d mkdir -p sandbox/proxy/conf.stream.d # pull and tag the sandbox and the proxy container docker pull "$registry/$name:$version" docker pull "$registry/$proxyName:$proxyVersion" # start the docker container and proxy if [ "$flavor" == "hdf" ]; then hostname="sandbox-hdf.hortonworks.com" elif [ "$flavor" == "hdp" ]; then hostname="sandbox-hdp.hortonworks.com" fi version=$(docker images | grep $registry/$name | awk '{print $2}'); # Create cda docker network docker network create cda 2>/dev/null # Deploy the sandbox into the cda docker network docker run --privileged --name $name -h $hostname --network=cda --network-alias=$hostname -d "$registry/$name:$version" echo " Remove existing postgres run files. Please wait" sleep 2 docker exec -t "$name" sh -c "rm -rf /var/run/postgresql/*; systemctl restart postgresql-9.6.service;" #Deploy the proxy container. sed 's/sandbox-hdp-security/sandbox-hdp/g' assets/generate-proxy-deploy-script.sh > assets/generate-proxy-deploy-script.sh.new mv -f assets/generate-proxy-deploy-script.sh.new assets/generate-proxy-deploy-script.sh chmod +x assets/generate-proxy-deploy-script.sh assets/generate-proxy-deploy-script.sh 2>/dev/null #check to see if it's windows if uname | grep MINGW; then sed -i -e 's/\( \/[a-z]\)/\U\1:/g' sandbox/proxy/proxy-deploy.sh fi chmod +x sandbox/proxy/proxy-deploy.sh 2>/dev/null sandbox/proxy/proxy-deploy.sh          LOG     ➜ Documents sh docker-deploy-hdp30.sh + registry=hortonworks + name=sandbox-hdp + version=3.0.1 + proxyName=sandbox-proxy + proxyVersion=1.0 + flavor=hdp + echo hdp + mkdir -p sandbox/proxy/conf.d + mkdir -p sandbox/proxy/conf.stream.d + docker pull hortonworks/sandbox-hdp:3.0.1 3.0.1: Pulling from hortonworks/sandbox-hdp 70799bbf2226: Pull complete 40963917cdad: Pull complete 3fe9adbb8d7e: Pull complete ee3ec4e8cb3d: Pull complete 7ea5917732c0: Pull complete 2d951411620c: Pull complete f4c5e354e7fd: Pull complete 22ffa6ef360f: Pull complete 2060aa0f3751: Pull complete ca01ba34744d: Pull complete 83326dded077: Pull complete eb3d71b90b73: Pull complete bdd1cab41c81: Pull complete 500cc770c4bd: Pull complete 0cb1decd5474: Pull complete b9591f4b6855: Pull complete f28e56086127: Pull complete e7de4e7d0bca: Pull complete ec77967d2166: Pull complete 4fdcae170114: Pull complete 6347f5df8ffc: Pull complete 6a6ecc232709: Pull complete ea845898ff50: Pull complete 02135573b1bf: Pull complete cb0176867cd8: Pull complete 3c08321268fd: Pull complete 82e82a97c465: Pull complete 8aaaa48ed101: Pull complete 74b321ac2ac5: Pull complete 569da02c0a66: Pull complete af40820407ef: Pull complete Digest: sha256:7b767af7b42030fb1dd0f672b801199241e6bef1258e3ce57361edb779d95921 Status: Downloaded newer image for hortonworks/sandbox-hdp:3.0.1 docker.io/hortonworks/sandbox-hdp:3.0.1 + docker pull hortonworks/sandbox-proxy:1.0 1.0: Pulling from hortonworks/sandbox-proxy 951bdea65c93: Pull complete 4b9047c5fbbb: Pull complete 773156407aae: Pull complete d8524176841d: Pull complete Digest: sha256:42e4cfbcbb76af07e5d8f47a183a0d4105e65a1e7ef39fe37ab746e8b2523e9e Status: Downloaded newer image for hortonworks/sandbox-proxy:1.0 docker.io/hortonworks/sandbox-proxy:1.0 + '[' hdp == hdf ']' + '[' hdp == hdp ']' + hostname=sandbox-hdp.hortonworks.com ++ docker images ++ grep hortonworks/sandbox-hdp ++ awk '{print $2}' + version=3.0.1 + docker network create cda ce52e1d04aa49c2da70dfd7405b95fc2e2184fed9625feaea0ddafb9ab5c361d + docker run --privileged --name sandbox-hdp -h sandbox-hdp.hortonworks.com --network=cda --network-alias=sandbox-hdp.hortonworks.com -d hortonworks/sandbox-hdp:3.0.1 WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 3ab0f0281918fc10d008c93aa7a844302e974b696e00788df76cde779fdec46d + echo ' Remove existing postgres run files. Please wait' Remove existing postgres run files. Please wait + sleep 2 + docker exec -t sandbox-hdp sh -c 'rm -rf /var/run/postgresql/*; systemctl restart postgresql-9.6.service;' Failed to get D-Bus connection: No such file or directory + sed s/sandbox-hdp-security/sandbox-hdp/g assets/generate-proxy-deploy-script.sh + mv -f assets/generate-proxy-deploy-script.sh.new assets/generate-proxy-deploy-script.sh + chmod +x assets/generate-proxy-deploy-script.sh + assets/generate-proxy-deploy-script.sh + uname + grep MINGW + chmod +x sandbox/proxy/proxy-deploy.sh + sandbox/proxy/proxy-deploy.sh WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested fbe0c31a680f51db9c91367df65493ea1cf874cc900466dd943824dc19bd9e74         # References   https://www.cloudera.com/tutorials/learning-the-ropes-of-the-hdp-sandbox.htmlhttps://javamana.com/2021/02/20210206191356735v.html   Is this issue have any solution or not possible to fix ?     ... View more