Hi ,
We have enabled KMS in our server, after enabling KMS and encryption , we are getting following error randomly , Is there anything we can try to resolve this issue ?
org.apache.spark.SparkException: Job aborted due to stage failure: Task 58 in stage 12.0 failed 4 times, most recent failure: Lost task 58.3 in stage 12.0 (TID 4151, lpdn0307.): java.io.IOException: java.lang.IllegalArgumentException: java.net.UnknownHostException: edhpen1262 at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:572) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:850) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:209) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:205) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:205) at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388) at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1440) at org.apache.hadoop.hdfs.DFSClient.createWrappedInputStream(DFSClient.java:1510)
We verified the DNS resolution and hostname , everything is looking good.
What else may be the issue.
Thanks,
Vishnu Ravi