Created 04-11-2025 06:38 PM
Unable to start dfsrouter daemon. Cluster is kerberized and using hadoop 3.3.4. Trying to implement hdfs router based federation. Configured hdfs-rbf-site.xml with required properties.
It is a working cluster. Zookeeper quorum is already configured in core-site.xml.
Below is the error from log. "Zookeeper connection string cannot be null".
ERROR router.FederationUtil - Could not instantiate: ZKDelegationTokenSecretManagerImpl
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hdfs.server.federation.router.FederationUtil.newInstance(FederationUtil.java:164)
at org.apache.hadoop.hdfs.server.federation.router.FederationUtil.newSecretManager(FederationUtil.java:224)
at org.apache.hadoop.hdfs.server.federation.router.security.RouterSecurityManager.<init>(RouterSecurityManager.java:60)
at org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.<init>(RouterRpcServer.java:293)
at org.apache.hadoop.hdfs.server.federation.router.Router.createRpcServer(Router.java:391)
at org.apache.hadoop.hdfs.server.federation.router.Router.serviceInit(Router.java:188)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.hdfs.server.federation.router.DFSRouter.main(DFSRouter.java:69)
Caused by: java.lang.NullPointerException: Zookeeper connection string cannot be null
at org.apache.hadoop.thirdparty.com.google.common.base.Preconditions.checkNotNull(Preconditions.java:899)
at org.apache.hadoop.security.token.delegation.ZKDelegationTokenSecretManager.<init>(ZKDelegationTokenSecretManager.java:168)
at org.apache.hadoop.hdfs.server.federation.router.security.token.ZKDelegationTokenSecretManagerImpl.<init>(ZKDelegationTokenSecretManagerImpl.java:42)
... 12 more
Any help is appreciated!
Created 08-19-2025 01:56 AM
Hi, @Hadoop16
The error Zookeeper connection string cannot be null means the Router process is expecting ZooKeeper configs for token management but isn’t finding them. Even if you already have a ZooKeeper quorum set in core-site.xml, the Router Federation requires its own configs in hdfs-rbf-site.xml. Specifically, you need to set hadoop.kms.authentication.zk-dt-secret-manager.zkConnectionString (or in some versions hadoop.security.token.service.use_ip + hadoop.zk.address) depending on your setup. Please double-check that hdfs-rbf-site.xml contains the federation and router related properties, including ZooKeeper connection string and Kerberos settings. Ensure the file is deployed to all Router nodes and included in the classpath. Also verify that the Router service user has Kerberos credentials and permissions to connect to ZooKeeper. Once the ZK connection string is set properly, the Router daemon should start without this error.
Created on 08-19-2025 07:10 AM - edited 08-19-2025 01:48 PM
@RAGHUY Thank you! I figured that later but Router starting to fail with below error.I have the jaas.conf in place. Any help on this is appreciated.