Support Questions
Find answers, ask questions, and share your expertise

How to distcp data between a HDP cluster and a MapR cluster?

Highlighted

How to distcp data between a HDP cluster and a MapR cluster?

New Contributor

I am using the following syntax: hadoop distcp hdfs://<SRC_CLUSTER>/<SRC_DIR> maprfs://<TGT_CLUSTER>:<PORT>/<TGT_DIR> . The statement fails with: Invalid arguments: java.lang.ClassNotFoundException: Class com.mapr.fs.MapRFileSystem not found. The HDP cluster is kerberized and the MapR cluster isn't. Can someone please help me here.

2 REPLIES 2

Re: How to distcp data between a HDP cluster and a MapR cluster?

Contributor
@Abhiram Gattamaneni

HDP distribution doesn't support Mapr Filesystem not it have Mapr class jars. The above error is nothing to do with Kerborized env. My suggestion will be having Mapr NFSmount on client node and use hadoop copyFromLocal and copy the file to HDFS and vise versa.

Highlighted

Re: How to distcp data between a HDP cluster and a MapR cluster?

New Contributor

This can also be achieved using webhdfs but make sure it is enabled.

Command for example

hadoop distcp maprfs:///path/tempdir webhdfs://hostname:50070/path/tempdir