pull docker images from docker hub (docker.io/hortonworks/sandbox-hdp)

[#1]

I tried to pull the docker image hortonworks/sandbox-hdp) from the docker hub (docker.io/hortonworks/sandbox-hdp). The described pull command fails ("docker pull hortonworks/sandbox-hdp"), as no 'latest' tag is set after the recent push (4 days ago), but "docker pull hortonworks/sandbox-hdp:2:6:5" or "docker pull hortonworks/sandbox-hdp:3.0.0" will do the pull)

Is that repository an official Hortonworks build, and am I supposed to pull a sandbox image from there? Or should i take another docker repository? Or is the only supported way the web download?

I am asking as the pull fails with an error "unauthorized: authentication required", but only after quite some time. Now I am trying to figure out if it is actually an authorization issue, or it might be a kind of timeout issue, which I would have to sort out on my environment (in many docker support queries this issue was sorted out by setting host time correctly).

[#2 · Accepted solution]

@Harald Berghoff

I checked the docker-deploy script for hdp 2.6.5 and we are doing docker pull from the hortonworks/sandbox-hdp in dockerhub.

However this deploy script is not just doing that. Having that said you might want to wait until the sandbox for 3.0 is added on the hortonworks portal along with the corresponding scripts & instructions.

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

[#3]

@Felix Albani thanks for the reply. So actually the repository is public and I have to check on my environment why it fails? The failure is from docker, so i don't think it is related to the software on the image itself.

[#4]

seemed to be an issue with my environment, today I was able to pull the image successfully:

# docker pull hortonworks/sandbox-hdp:3.0.0
Trying to pull repository docker.io/hortonworks/sandbox-hdp ... 
sha256:3e75083ce3b38475e6825ab24a09adba229b9d67b0be2218f0ce385dcfa0452a: Pulling from docker.io/hortonworks/sandbox-hdp
70799bbf2226: Pulling fs layer 
[...]
055f548f8d84: Pull complete 
Digest: sha256:3e75083ce3b38475e6825ab24a09adba229b9d67b0be2218f0ce385dcfa0452a
Status: Downloaded newer image for docker.io/hortonworks/sandbox-hdp:3.0.0

[#5]

The correct way to docker pull repository is

docker pull "$registry/$name:$version"

hence the correct command to pull docker image is

docker pull hortonworks/sandbox-hdp:2:6:5

docker deploy script is also using above command to pull the sandbox image so when you are using so it is official command to pull the sandbox image "docker pull hortonworks/sandbox-hdp:2:6:5"

[#6]

@Pankaj Kadam the command to pull is fine, you are right that the version is part of the image name to be pulled. It is just that if you ommit the version it defaults to 'latest', which is typically also maintained as a tag in the docker hub. So there is nothing wrong with the image, it's just unusual to not maintain a 'latest' flag.

[#7]

:~/HDP# sh docker-deploy-hdp30.sh

+ registry=hortonworks

+ name=sandbox-hdp

+ version=3.0.1

+ proxyName=sandbox-proxy

+ proxyVersion=1.0

+ flavor=hdp

+ echo hdp

+ mkdir -p sandbox/proxy/conf.d

+ mkdir -p sandbox/proxy/conf.stream.d

+ docker pull hortonworks/sandbox-hdp:3.0.1

3.0.1: Pulling from hortonworks/sandbox-hdp

70799bbf2226: Downloading [================================> ] 47.17MB/72.85MB

40963917cdad: Pulling fs layer

70799bbf2226: Downloading [===================================> ] 51.45MB/72.85MB

ee3ec4e8cb3d: Downloading [=======> ] 57.28MB/368.8MB

ee3ec4e8cb3d: Downloading [=======> ] 57.81MB/368.8MB

ee3ec4e8cb3d: Downloading [========> ] 64.24MB/368.8MB

7ea5917732c0: Download complete

2d951411620c: Download complete

f4c5e354e7fd: Downloading

ca01ba34744d: Waiting

83326dded077: Waiting

eb3d71b90b73: Waiting

bdd1cab41c81: Waiting

500cc770c4bd: Waiting

0cb1decd5474: Waiting

b9591f4b6855: Waiting

f28e56086127: Waiting

e7de4e7d0bca: Waiting

ec77967d2166: Waiting

4fdcae170114: Waiting

6347f5df8ffc: Waiting

6a6ecc232709: Waiting

ea845898ff50: Waiting

02135573b1bf: Waiting

cb0176867cd8: Waiting

3c08321268fd: Pulling fs layer

82e82a97c465: Waiting

3c08321268fd: Waiting

74b321ac2ac5: Waiting

569da02c0a66: Waiting

af40820407ef: Pulling fs layer

af40820407ef: Waiting

unauthorized: authentication required

+ docker pull hortonworks/sandbox-proxy:1.0

1.0: Pulling from hortonworks/sandbox-proxy

Digest: sha256:42e4cfbcbb76af07e5d8f47a183a0d4105e65a1e7ef39fe37ab746e8b2523e9e

Status: Image is up to date for hortonworks/sandbox-proxy:1.0

+ [ hdp = hdf ]

+ [ hdp = hdp ]

+ hostname=sandbox-hdp.hortonworks.com

+ docker images

+ grep hortonworks/sandbox-hdp

+ awk {print $2}

+ version=

+ docker network create cda

+ docker run --privileged --name sandbox-hdp -h sandbox-hdp.hortonworks.com --network=cda --network-alias=sandbox-hdp.hortonworks.com -d hortonworks/sandbox-hdp:

docker: invalid reference format.

See 'docker run --help'.

+ echo Remove existing postgres run files. Please wait

Remove existing postgres run files. Please wait

+ sleep 2

+ docker exec -t sandbox-hdp sh -c rm -rf /var/run/postgresql/*; systemctl restart postgresql-9.6.service;

Error: No such container: sandbox-hdp

+ sed s/sandbox-hdp-security/sandbox-hdp/g assets/generate-proxy-deploy-script.sh

+ mv -f assets/generate-proxy-deploy-script.sh.new assets/generate-proxy-deploy-script.sh

+ chmod +x assets/generate-proxy-deploy-script.sh

+ assets/generate-proxy-deploy-script.sh

+ uname

+ grep MINGW

+ chmod +x sandbox/proxy/proxy-deploy.sh

+ sandbox/proxy/proxy-deploy.sh

sandbox-proxy

baed363d86a7a73e7712b838df09c2aeda99a6b15f0333e9f8e6f445902a383b

docker: Error response from daemon: driver failed programming external connectivity on endpoint sandbox-proxy (d53fe3e3eba2b296b2f6acaa6d8202732308d93cd987157130dfdddef1b82170): Error starting userland proxy: listen tcp 0.0.0.0:8042: bind: address already in use.

i have gone through the script,:- it means docker pull hortonworks/sandbox-hdp:3:0:1

cat docker-deploy-hdp30.sh

# CAN EDIT THESE VALUES

registry="hortonworks"

name="sandbox-hdp"

version="3.0.1"

proxyName="sandbox-proxy"

proxyVersion="1.0"

flavor="hdp"

# NO EDITS BEYOND THIS LINE

# housekeeping

echo $flavor > sandbox-flavor

# create necessary folders for nginx and copy over our rule generation script there

mkdir -p sandbox/proxy/conf.d

mkdir -p sandbox/proxy/conf.stream.d

# pull and tag the sandbox and the proxy container

docker pull "$registry/$name:$version"........

========

but it fails with the error "unauthorized: authentication required". even though if i executed

docker pull hortonworks/sandbox-hdp:3:0:1 also i had logged into docker with docker login -u <userid> -p <password>