Reply
Highlighted
New Contributor
Posts: 2
Registered: ‎06-06-2016

Spark Program throwing error java.net.UnknownHostException

Hi ,

 

We have enabled KMS in our server, after enabling KMS and encryption , we are getting following error randomly , Is there anything we can try to resolve this issue ?

 

org.apache.spark.SparkException: Job aborted due to stage failure: Task 58 in stage 12.0 failed 4 times, most recent failure: Lost task 58.3 in stage 12.0 (TID 4151, lpdn0307.): java.io.IOException: java.lang.IllegalArgumentException: java.net.UnknownHostException: edhpen1262 at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:572) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:850) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:209) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:205) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:205) at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388) at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1440) at org.apache.hadoop.hdfs.DFSClient.createWrappedInputStream(DFSClient.java:1510)

 

We verified the DNS resolution and hostname , everything is looking good.

 

What else may be the issue.

 

Thanks,

Vishnu Ravi

Cloudera Employee
Posts: 8
Registered: ‎01-27-2017

Re: Spark Program throwing error java.net.UnknownHostException

Hello, 

 

Can you provide what versions of CDH/CM you are currently running? Have any other services/componets been impacted after enabling KMS? 

 

Thanks, 

Jordan 

Announcements