Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Resource Localization specific to a container-Hadoop Yarn

Resource Localization specific to a container-Hadoop Yarn

New Contributor

I have a custom yarn application which is intended to launch few containers (2 or more). In each container it is intended to launch a bash script (customized server) so that when all the containers are launched , there will be equivalent number of servers running on. The zip file containing bash script and all the required files to run the server is resides on the HDFS.

When launching the containers , ApplicationMaster is written such that,each container has a its own copy of the zip file. In the resource localization for the zip file, I have given Resource Visibility as APPLICATION and Resource Type as ARCHIVE.

The problem is when the ApplicationMaster launch the containers, unarchived zip file in all the containers refer to the same folder. Due to that changes done in one folder in one container is visible to all the other containers. But what I need is zip file in the HDFS being localized to containers such that, the resource (unarchived file) is specific to a container. so that changes in one container is not visible to other containers. How can this be achieved., Thank you.

1 REPLY 1

Re: Resource Localization specific to a container-Hadoop Yarn

New Contributor

@Rameshka Amarasinghe I am currently facing the same problem.

Could you find a solution at the end?