Found it. I would have left it like that however since it in Production and sometimes you just need to know. For replication, we Falcon needs to be aware of local and remote Cluster ID via below property.
dfs.nameservices mine has value like "prodID, devID" What balancer does is it tries to reach both the nameservices and if I'm running the command in prod cluster as hdfs with proper ticket, it will throw a errror "hdfs-prod" (which is my principal w/o REALM) however it is still balancing the prod cluster, so the error although not clear is actually permission denied on remote name service (which makes sense) as the user still is "hdfs" however the principal is different "hdfs-dev" in my case. I ran the same command in Dev and cluster wwas rebalanced however I got the same error, this time Access denied for user hdfs-dev. Superuser privilege is required. Thanks for support @emaxwell, @mqureshi, @Kuldeep Kulkarni. I hope above answer will help others. (few of other hdfs commands have no effect/errors) Thanks Mayank
... View more
