Kubernetes Yarn integration

[#1]

Below page explains the features that Kubernetes Yarn integration offers.

https://hortonworks.com/blog/docker-kubernetes-apache-hadoop-yarn/

However, the tutorial it refers only gives instructions to install a local cluster.

Do you know how to integrate Kubernetes with existing Yarn cluster?

Alternative or a second question would be:

Is that advisable / rational to run Kubernetes cluster in the same physical cluster that Hadoop runs?

Thanks in advance...

[#2]

Hi Sedat, will you able to share the use case that you're trying to solve? e.g. a singe cluster of compute resources?

[#3]

Hi @Saumitra Buragohain

Yes. I want to run my custom applications (web service with background compute) and / or applications like TensorFlow on my existing Hadoop cluster which has good enough processing capacity. I understand that running them on a docker cluster seems to be the best way to get the best performance. At this point I have the need of resource planning. I want to delegate scheduling of Kubernetes to Yarn but don't know how to do this.

I know there is also docker container executor class support released with Hadoop 2.7.3 but I think this will switch all containers to docker (maybe even my custom) containers.

[#4]

Follow-up post...

Can anybody give any idea?