Hi, I am trying to run a spark job from a separate VM using an HDI clusters YARN. So I have copied all the spark conf files from the HDI cluster to the VM so that it accesses the YARN of the HDI cluster and not the local YARN but now I get the following exception:
Client [pool-15-thread-1] - Exception encountered while connecting to the server : java.lang.IllegalArgumentException: Server has invalid Kerberos principal: rm/<name-node hostname>@xyz
Its a warning but it keeps coming up and the job doesnt proceed.
I tried copying the rm/<name-node hostname>@xyz keytab file from the head node to the VM but it still throws the same exception. The name node hostname is resolved by the VM so it's not that issue.
Could anyone help me figure out how to solve this?