Issue with HDFS Encryption

[#1]

Hi,

I am trying to distcp data between two encryption zones located on two different clusters. Data has been copied successfully. However, when I read the data on the target cluster, I see some gibberish being printed on the terminal.

Encryption Zone on source has been created with key (test-key). As its a DR requirement, I created a key on the target cluster with the same key name i.e. test-key. However, fundamentally they both are completely independent clusters.

I presume when DistCp reads the data from the source cluster, it should read and transfer the data transparently using source side key and material and then write to target using the target’s key and material

Wondering where this has gone wrong. Any pointers?

[#2 · Accepted solution]

@Vijaya Narayana Reddy Bhoomi Reddy

You have to export and import the key as well. Just creating the key as the same name does not make it the same key.

That is the reason you are seeing gibberish values.

I wrote an article to automate this task with the automation script link. You can just change the cluster inside the script and change the directory locations(if any) to make it work.

https://community.hortonworks.com/content/kbentry/110144/hdfs-encrypted-zone-intra-cluster-transfer-...