Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Can Yarn on CDH 5.7 launch (docker) containers running servers?

avatar
Explorer

Hello:

 

We have some CDH 5.7 clusters, and we want to deploy servers in containers to the nodes (probably via yarn).

Informally, we have some workflows that contact remote servers to do a fairly complex computation, and are interested in using cluster nodes to host these servers (to promote more elastic use of our hardware).   One approach we are entertaining is to attempt to package these servers in containers (docker is the current candidate), and we believe that some work has been done to launch docker containers via Yarn (e.g. https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/DockerContainerExecutor.html).  My questions are:

  • Is it possible to deploy user configured containers (e.g. docker) to deploy servers via Yarn?
  • If so, would docker be a suitable container technology for this use case?  If not docker, what would be better?
  • If we were to use docker containers for this use case, would all jobs in the system need to use docker containers?  How does that affect the build process and (oozie based) deployment?

Thanks:

 

Bill

1 ACCEPTED SOLUTION

avatar
Contributor
There is no plan to support Docker with CDH at the moment.

View solution in original post

3 REPLIES 3

avatar
Explorer

Hello: 

 

My apologoies, there is an error in the subject, it should have read CDH 5.7 not CDH 7.5.   I'm trying to fix it.

 

Also, if a newer version of CDH has support (or if it is on the road map) I'd be interested in hearing about it.

 

With best regards:

 

Bill M.

avatar
Contributor
There is no plan to support Docker with CDH at the moment.

avatar
New Contributor

So this is no longer supported? Why -- technical or business reasons? Anyway, maybe you could update it to make that clear?

https://hub.docker.com/r/cloudera/quickstart/

How about clusterdock, Cloudera blogged about it last year:

http://blog.cloudera.com/blog/2016/08/multi-node-clusters-with-cloudera-quickstart-for-docker/

Will this image be supported as an easy Production-like environment?

https://hub.docker.com/r/cloudera/clusterdock/

 

Thanks,

Jamshid