2 Doubts - Distcp between secure clusters in different kerberos realms -

Question

Hi Community Team.

I´m configuring Distcp between secure clusters in different kerberos realms.

Source= HDP 2.6.5

Target= CDP Private Cloud Base 7.1.5

I am setting up, but in the process I have doubts as to how these 2 steps are done, I add the links that show little detail

1)First doubt (what is the step by step of this):
https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/scaling-namespaces/topics/hdfs-distcp-truststore-properties.html

2)Second doubt (what is the step by step of this):
https://docs.cloudera.com/cdp-private-cloud-base/7.1.5/scaling-namespaces/topics/hdfs-distcp-set-hadoop-conf.html

thanking the help

willx · Accepted Answer

Hi @vciampa,

You can refer to this doc for migrating data from secured HDP to secured CDP:

https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/data-migration/topics/rm-migrate-securehdp-securecdp-distcp.html

For question 2)

I think it says about to set hadoop configuration file location to Destination cluster if you use HFTP(refer to HFTP for explanation), if not using HFTP, you should set it to Source cluster. But I think this link is kinda confusing it looks like HDP documents not CDP documents.

There are different HADOOP_CONF location between HDP and CDP.

https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/concepts/topics/cm-server-client-configuration.html

For example,

In HDP, HADOOP_CONF location is /etc/hadoop/conf, it contains both service/client sides configuration files.

In CDP, Cloudera manager separate the client and server configurations:

Client configurations located at /etc/hadoop/conf/hdfs-site.xml

Server configurations located in the latest process directory of each role: under /var/run/cloudera-scm-agent/process/<latest_pid>-hdfs-NAMENODE/hdfs-site.xml

As for question 1) please follow the answer of @vidanimegh in the previous post.

As for question 2) please just ignore this step and Follow the first link I provided in this post.

Thanks,

Will

If the answer helps, please accept as solution and click thumbs up.

vidanimegh · Answer

Hi @vciampa ,

For the first question, Go into the HDFS configuration in CDP, and search for "SSL Client".

Add the properties given in the link shared by you under "HDFS Advanced Configuration Snippet (Safety Valve) for ssl-client.xml".

Also, you will see "Cluster-Wide Default TLS/SSL Client Truststore Location" and "Cluster-Wide Default TLS/SSL Client Truststore Password". Set these values accordingly to your Truststore location and Truststore password.

Not sure about question 2 at the moment, will let you know if I have some info.

Thanks,

Megh

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.