Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Has anyone done distcp between secured clusters but different REALM?

avatar
2 ACCEPTED SOLUTIONS

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
4 REPLIES 4

avatar

I have not done distcp with different Kerberos REALMS, but I think this should be possible. Our documentation only mentions "same principal name must be assigned to the applicable NameNodes", so that auth_to_local configuration can calculate the same username on both sides (Kerberos principal: nn/host1@realm will be user "nn"). As long as the different realms use the same KDC or the KDCs trust each other, this should be possible.

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
@sprakash

The fact that distcp works with some configurations indicates you probably have Security set up right, as well as giving you an obvious work-around. To try to answer your question, please provide some clarifying information:

  1. When you speak of mapred-client.xml, do you mean mapred-site.xml on the client machine?
  2. When you speak of changing the framework, do you mean the "mapreduce.framework.name" configuration parameter in mapred-side.xml?
  3. Do you change it only on the client machine, or throughout both clusters?
  4. The allowed values of that parameter are "local", "classic", and "yarn". When you change it to not be "yarn", what do you set it to?
  5. Do you have "mapreduce.application.framework.path" set? If so, to what value?