Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark Program throwing error java.net.UnknownHostException

Spark Program throwing error java.net.UnknownHostException

New Contributor

Hi ,

 

We have enabled KMS in our server, after enabling KMS and encryption , we are getting following error randomly , Is there anything we can try to resolve this issue ?

 

org.apache.spark.SparkException: Job aborted due to stage failure: Task 58 in stage 12.0 failed 4 times, most recent failure: Lost task 58.3 in stage 12.0 (TID 4151, lpdn0307.): java.io.IOException: java.lang.IllegalArgumentException: java.net.UnknownHostException: edhpen1262 at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:572) at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:850) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:209) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:205) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:94) at org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:205) at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388) at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1440) at org.apache.hadoop.hdfs.DFSClient.createWrappedInputStream(DFSClient.java:1510)

 

We verified the DNS resolution and hostname , everything is looking good.

 

What else may be the issue.

 

Thanks,

Vishnu Ravi

2 REPLIES 2

Re: Spark Program throwing error java.net.UnknownHostException

Contributor

Hello, 

 

Can you provide what versions of CDH/CM you are currently running? Have any other services/componets been impacted after enabling KMS? 

 

Thanks, 

Jordan 

Re: Spark Program throwing error java.net.UnknownHostException

New Contributor

Did anyone found the solution for this issue? I have a similar issue, We are using CM5.11 and installed Spark2 seperately on other server. When we enable KMS on the cluster then Spark2 throws Unknown Host exception.

Don't have an account?
Coming from Hortonworks? Activate your account here