Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Ranger KMS failed to start

Highlighted

Ranger KMS failed to start

Expert Contributor

Hi,

We have installed luna client in Ranger KMS server and able to export the master key to HSM server from Ranger DB. But, after enabling KMS HSM, Ranger KMS is not getting started and getting below error message.

HSM properties.

ranger.ks.hsm.enabled=true
ranger.ks.hsm.partition.name=ranger
ranger.ks.hsm.partition.password=*******
ranger.ks.hsm.type=LunaProvider

kms.log

2018-11-20 07:33:24,049 ERROR RangerKMSDB - DB Flavor could not be determined
2018-11-20 07:33:25,018 INFO  RangerKMSDB - Connected to DB : true
2018-11-20 07:33:25,019 INFO  RangerHSM - RangerHSM provider
2018-11-20 07:33:25,245 INFO  AuditProviderFactory - ==> JVMShutdownHook.run()
2018-11-20 07:33:25,245 INFO  AuditProviderFactory - JVMShutdownHook: Signalling async audit cleanup to start.
2018-11-20 07:33:25,245 INFO  AuditProviderFactory - RangerAsyncAuditCleanup: Starting cleanup
2018-11-20 07:33:25,246 INFO  AuditAsyncQueue - Stop called. name=kms.async
2018-11-20 07:33:25,246 INFO  AuditAsyncQueue - Interrupting consumerThread. name=kms.async, consumer=kms.async.summary
2018-11-20 07:33:25,246 INFO  AuditProviderFactory - RangerAsyncAuditCleanup: Done cleanup
2018-11-20 07:33:25,246 INFO  AuditProviderFactory - RangerAsyncAuditCleanup: Waiting to audit cleanup start signal
2018-11-20 07:33:25,246 INFO  AuditAsyncQueue - Caught exception in consumer thread. Shutdown might be in progress
2018-11-20 07:33:25,246 INFO  AuditAsyncQueue - Exiting polling loop. name=kms.async
2018-11-20 07:33:25,246 INFO  AuditAsyncQueue - Calling to stop consumer. name=kms.async, consumer.name=kms.async.summary
2018-11-20 07:33:25,246 INFO  AuditSummaryQueue - Stop called. name=kms.async.summary
2018-11-20 07:33:25,246 INFO  AuditSummaryQueue - Interrupting consumerThread. name=kms.async.summary, consumer=kms.async.summary.multi_dest
2018-11-20 07:33:25,246 INFO  AuditAsyncQueue - Exiting consumerThread.run() method. name=kms.async
2018-11-20 07:33:25,247 INFO  AuditSummaryQueue - Caught exception in consumer thread. Shutdown might be in progress
2018-11-20 07:33:25,247 INFO  AuditProviderFactory - JVMShutdownHook: Waiting up to 30 seconds for audit cleanup to finish.
2018-11-20 07:33:25,247 INFO  AuditSummaryQueue - Exiting polling loop. name=kms.async.summary
2018-11-20 07:33:25,247 INFO  AuditSummaryQueue - Calling to stop consumer. name=kms.async.summary, consumer.name=kms.async.summary.multi_dest
2018-11-20 07:33:25,247 INFO  AuditProviderFactory - JVMShutdownHook: Audit cleanup finished after 1 milli seconds
2018-11-20 07:33:25,247 INFO  AuditProviderFactory - JVMShutdownHook: Interrupting ranger async audit cleanup thread
2018-11-20 07:33:25,247 INFO  AuditProviderFactory - <== JVMShutdownHook.run()
2018-11-20 07:33:25,247 INFO  AuditProviderFactory - RangerAsyncAuditCleanup: Interrupted while waiting for audit startCleanup signal!  Exiting the thread...
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
        at java.util.concurrent.Semaphore.acquire(Semaphore.java:312)
        at org.apache.ranger.audit.provider.AuditProviderFactory$RangerAsyncAuditCleanup.run(AuditProviderFactory.java:487)
        at java.lang.Thread.run(Thread.java:748)


 

catalina.out

java.io.IOException: Keystore was tampered with, or password was incorrect
        at org.apache.hadoop.crypto.key.RangerKeyStore.engineLoad(RangerKeyStore.java:361)
        at org.apache.hadoop.crypto.key.RangerKeyStoreProvider.loadKeys(RangerKeyStoreProvider.java:151)
        at org.apache.hadoop.crypto.key.RangerKeyStoreProvider.reloadKeys(RangerKeyStoreProvider.java:382)
        at org.apache.hadoop.crypto.key.RangerKeyStoreProvider.<init>(RangerKeyStoreProvider.java:111)
        at org.apache.hadoop.crypto.key.RangerKeyStoreProvider$Factory.createProvider(RangerKeyStoreProvider.java:399)
        at org.apache.hadoop.crypto.key.KeyProviderFactory.get(KeyProviderFactory.java:95)
        at org.apache.hadoop.crypto.key.kms.server.KMSWebApp.contextInitialized(KMSWebApp.java:176)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5118)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5634)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1571)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1561)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.security.UnrecoverableKeyException: Password verification failed
        at org.apache.hadoop.crypto.key.RangerKeyStore.engineLoad(RangerKeyStore.java:359)
        ... 15 more

ERROR: Hadoop KMS could not be started

REASON: java.lang.NullPointerException
Stacktrace:
---------------------------------------------------
java.lang.NullPointerException
        at org.apache.hadoop.crypto.key.kms.server.KMSWebApp.contextInitialized(KMSWebApp.java:178)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5118)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5634)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1571)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1561)

Could you please help on this issue.

@Akhil S Naik

Thank you,.