Unable to Access kerberized cluster from my mac machine. Getting 401 Authentication error in curl

[#1]

kerberose.pngI am trying to access the cluster that i created in AWS from my mac. I am getting the below error when i do curl.This is a secured Kerberized cluster(MIT KDC)

[#2]

Did you obtain a ticket first, and is krb5.conf configured to use the same KDC on your laptop and the cluster? Any errors in the namenode log?

[#3]

1. Copied the krb5.conf from cluster to /etc on mac.

2. Created a keytab file and ftpied the file to mac.

3 Provided 664 permission to the keytab file

4. Ticket is granted to me when i do kinit from mac. Klist does show valid ticket being granted.

5. I run into 401 error when i do curl after doing kinit . When i drill down into the error i see the error code gss_init_sec_context() failed: unknown mech-code 0 for mech unknown

6. I don't see any error being logged in namenode. Looks like the curl command from mac is not hitting the cluster .

7. tcsdump on namenode port(50070) doesn't shows any call being made from mac.

Note : - I am on HDP2.3 and have namenode HA. I am able to do curl from one of the node within the cluster using the same keytab file and curl command.

[#4]

Can you confirm network connectivity from Mac to namenode by some other means, like hdfs client (not webhdfs) or ping/ssh? Do you see the same behavior from both NNs?

[#5]

@Alex Miller I am able to ssh to namenode. I use the below command to ssh to namenode.

ssh -i sumit.pem ec2-use@public-hostname-dns.

[#6]

Can you confirm that you have a valid tgt ? Klist should show that.

[#7]

Yes @Saurabh Jain klist shows a valid ticket.

[#8]

@Alex Miller

1. Copied the krb5.conf from cluster to /etc on mac.

2. Created a keytab file and ftpied the file to mac.

3 Provided 664 permission to the keytab file

4. Ticket is granted to me when i do kinit from mac. Klist does show valid ticket being granted.

5. I run into 401 error when i do curl after doing kinit . When i drill down into the error i see the error code gss_init_sec_context() failed: unknown mech-code 0 for mech unknown

6. I don't see any error being logged in namenode. Looks like the curl command from mac is not hitting the cluster .

7. tcsdump on namenode port(50070) doesn't shows any call being made from mac.

Note : - I am on HDP2.3 and have namenode HA. I am able to do curl from one of the node within the cluster using the same keytab file and curl command.

[#9]

what is the exact curl command you are running?

[#10]

I am using the below curl command.

curl -iv -u: --negotiate http://ec2-52-33-77-118.us-west-2.compute.amazonaws.com:50070/webhdfs/v1/?op=LISTSTATUS

this command works when i login to any node in the cluster and try to curl . Where as when i do the same from mac i get 401 error code --> gss_init_sec_context() failed: unknown mech-code 0 for mech unknown.

Do we need to do anything else apart from copying the krb5.conf to /etc and kinit to get the ticket?