You are right but what I am trying to say is if thousands of applications concurrently tries to modify the same object at the same time , we have to lock that object (which is being modified) just to make it synchronous. Do you agree? This can lead to some of the request to starve and thats what we are seeing with S3a client. A hadoop copy operation has two parts.... put a temp file and then rename it. So the put operation is successful, we can see a temp file uploaded to our object store but some of this renaming requests TimesOut because we see multiple requests from S3A client trying to modify the same object and some of them are not able to acquire lock and times out.
... View more
