Support Questions
Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Resource Localization specific to a container-Hadoop Yarn

New Contributor

I have a custom yarn application which is intended to launch few containers (2 or more). In each container it is intended to launch a bash script (customized server) so that when all the containers are launched , there will be equivalent number of servers running on. The zip file containing bash script and all the required files to run the server is resides on the HDFS.

When launching the containers , ApplicationMaster is written such that,each container has a its own copy of the zip file. In the resource localization for the zip file, I have given Resource Visibility as APPLICATION and Resource Type as ARCHIVE.

The problem is when the ApplicationMaster launch the containers, unarchived zip file in all the containers refer to the same folder. Due to that changes done in one folder in one container is visible to all the other containers. But what I need is zip file in the HDFS being localized to containers such that, the resource (unarchived file) is specific to a container. so that changes in one container is not visible to other containers. How can this be achieved., Thank you.


New Contributor

@Rameshka Amarasinghe I am currently facing the same problem.

Could you find a solution at the end?