Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can Yarn on CDH 5.7 launch (docker) containers running servers?

Solved Go to solution

Can Yarn on CDH 5.7 launch (docker) containers running servers?

Contributor

Hello:

 

We have some CDH 5.7 clusters, and we want to deploy servers in containers to the nodes (probably via yarn).

Informally, we have some workflows that contact remote servers to do a fairly complex computation, and are interested in using cluster nodes to host these servers (to promote more elastic use of our hardware).   One approach we are entertaining is to attempt to package these servers in containers (docker is the current candidate), and we believe that some work has been done to launch docker containers via Yarn (e.g. https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/DockerContainerExecutor.html).  My questions are:

  • Is it possible to deploy user configured containers (e.g. docker) to deploy servers via Yarn?
  • If so, would docker be a suitable container technology for this use case?  If not docker, what would be better?
  • If we were to use docker containers for this use case, would all jobs in the system need to use docker containers?  How does that affect the build process and (oozie based) deployment?

Thanks:

 

Bill

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Can Yarn on CDH 5.7 launch (docker) containers running servers?

Cloudera Employee
There is no plan to support Docker with CDH at the moment.
3 REPLIES 3

Re: Can Yarn on CDH 5.7 launch (docker) containers running servers?

Contributor

Hello: 

 

My apologoies, there is an error in the subject, it should have read CDH 5.7 not CDH 7.5.   I'm trying to fix it.

 

Also, if a newer version of CDH has support (or if it is on the road map) I'd be interested in hearing about it.

 

With best regards:

 

Bill M.

Re: Can Yarn on CDH 5.7 launch (docker) containers running servers?

Cloudera Employee
There is no plan to support Docker with CDH at the moment.
Highlighted

Re: Can Yarn on CDH 5.7 launch (docker) containers running servers?

New Contributor

So this is no longer supported? Why -- technical or business reasons? Anyway, maybe you could update it to make that clear?

https://hub.docker.com/r/cloudera/quickstart/

How about clusterdock, Cloudera blogged about it last year:

http://blog.cloudera.com/blog/2016/08/multi-node-clusters-with-cloudera-quickstart-for-docker/

Will this image be supported as an easy Production-like environment?

https://hub.docker.com/r/cloudera/clusterdock/

 

Thanks,

Jamshid

Don't have an account?
Coming from Hortonworks? Activate your account here