Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to distcp data between a HDP cluster and a MapR cluster?

Highlighted

How to distcp data between a HDP cluster and a MapR cluster?

New Contributor

I am using the following syntax: hadoop distcp hdfs://<SRC_CLUSTER>/<SRC_DIR> maprfs://<TGT_CLUSTER>:<PORT>/<TGT_DIR> . The statement fails with: Invalid arguments: java.lang.ClassNotFoundException: Class com.mapr.fs.MapRFileSystem not found. The HDP cluster is kerberized and the MapR cluster isn't. Can someone please help me here.

2 REPLIES 2

Re: How to distcp data between a HDP cluster and a MapR cluster?

Contributor
@Abhiram Gattamaneni

HDP distribution doesn't support Mapr Filesystem not it have Mapr class jars. The above error is nothing to do with Kerborized env. My suggestion will be having Mapr NFSmount on client node and use hadoop copyFromLocal and copy the file to HDFS and vise versa.

Re: How to distcp data between a HDP cluster and a MapR cluster?

New Contributor

This can also be achieved using webhdfs but make sure it is enabled.

Command for example

hadoop distcp maprfs:///path/tempdir webhdfs://hostname:50070/path/tempdir

Don't have an account?
Coming from Hortonworks? Activate your account here