Upscale/Downscale of the emr cluster corrupts hdfs jars /apps/tez/tez.tar.gz

[#1]

I am facing an issue in which the JARs kept in hdfs:///apps/tez/tez/tar.gz gets corrupted whenever I do emr cluster resizing after setting the property in hive-site.xml:

set hive.execution.engine=tez 

(with default file system as hdfs://)

How do I resolve this issue? Can anyone suggest?

[#2]

[Adding details to the original case]

To rectify this issue I tried refering the tar file directly from S3 without changing fs.defaultFS to s3://bucketName.

I did the following changes in tez-site.xml:

• set tez.lib.uris property to s3://bucketName

In hive-site.xml:

•  set hive.execution.engine=tez
• fs.defaultFS=hdfs:// 

But  still it is searching for tar file in hdfs.

S3 location where tez.tar.gz has been stored: s3://srev-emr-dev/F129/tez/tez.tar.gz where s3 bucket name is 'srev-emr-dev'

Facing below error:

java.io.FileNotFoundException: File does not exist: /F129/tez/tez.tar.gz

java.io.FileNotFoundException: File does not exist: /F129/tez/tez.tar.gz at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1441) ~[hadoop-hdfs-client-2.8.3-amzn-1.jar:?] at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1434) ~[hadoop-hdfs-client-2.8.3-amzn-1.jar:?] at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[hadoop-common-2.8.3-amzn-1.jar:?] at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1449) ~[hadoop-hdfs-client-2.8.3-amzn-1.jar:?] at org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(TezClientUtils.java:1031) ~[tez-api-0.8.4.jar:0.8.4] at org.apache.tez.client.TezClientUtils.addLocalResources(TezClientUtils.java:275) ~[tez-api-0.8.4.jar:0.8.4] at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:183) ~[tez-api-0.8.4.jar:0.8.4] at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:1057) ~[tez-api-0.8.4.jar:0.8.4] at org.apache.tez.client.TezClient.start(TezClient.java:447) ~[tez-api-0.8.4.jar:0.8.4] at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:376) ~[hive-exec-2.3.2-amzn-0.jar:2.3.2-amzn-0] at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.access$000(TezSessionState.java:87) ~[hive-exec-2.3.2-amzn-0.jar:2.3.2-amzn-0] at org.apache.hadoop.hive.ql.exec.tez.TezSessionState$1.call(TezSessionState.java:330) [hive-exec-2.3.2-amzn-0.jar:2.3.2-amzn-0] at org.apache.hadoop.hive.ql.exec.tez.TezSessionState$1.call(TezSessionState.java:326) [hive-exec-2.3.2-amzn-0.jar:2.3.2-amzn-0] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_222] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]