New Contributor
Posts: 3
Registered: ‎08-10-2015 when accessing CDH4 hdfs from CDH5 cluser nodes.

Hi ,

I am trying to distcp between CDH4(4.7.1) and CDH5(5.5.1) clusters. distcp is running into issues, please see below log on distcp command run.  Alternatively i tried to access CDH4 hdfs from CDH4 client it throwing below error.  Any help is appreciated.


simple hdfs access:


$ hadoop version
Hadoop 2.6.0-cdh5.5.1
Subversion -r e1581abbb6ab62b0a41b7ce6141d7280bf0c53da
Compiled by jenkins on 2015-12-02T18:38Z
Compiled with protoc 2.5.0
From source with checksum 98e07176d1787150a6a9c087627562c
This command was run using /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/jars/hadoop-common-2.6.0-cdh5.5.1.jar


$ hadoop fs -ls hdfs://
ls: Failed on local exception: Protocol message contained an invalid tag (zero).; Host Details : local host is: ""; destination host is: "":9000;


Below is distcp failure which run from CDH4 environment to push data to CDH5:


$ hadoop version
Hadoop 2.0.0-mr1-cdh4.7.1
Subversion file:///data/jenkins/workspace/generic-package-centos64-5-5/topdir/BUILD/hadoop-2.0.0-cdh4.7.1/src/hadoop-mapreduce1-project -r Unknown
Compiled by jenkins on Tue Nov 18 08:20:05 PST 2014
From source with checksum 9ff165dd3cdf37f78b5950cfcef45687
This command was run using /server/turn/deploy/build-160203/turn/lib/hadoop-common-2.0.0-cdh4.7.1.jar



$ hadoop distcp h hdfs://
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-4.7.1-1.cdh4.7.1.p0.47/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]

SLF4J: See for an explanation.
With failures, global counters are inaccurate; consider running with -i
Copy failed: Failed on local exception: Message missing required fields: callId, status; Host Details : local host is: ""; destination host is: "":8020;
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(
    at com.sun.proxy.$Proxy8.getFileInfo(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
    at java.lang.reflect.Method.invoke(
    at com.sun.proxy.$Proxy8.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(
    at org.apache.hadoop.hdfs.DFSClient.getFileInfo(
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(
    at org.apache.hadoop.fs.FileSystem.exists(
Caused by: Message missing required fields: callId, status
    at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto$Builder.buildParsed(
    at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto$Builder.access$1300(
    at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(
    at org.apache.hadoop.ipc.Client$Connection.receiveResponse(
    at org.apache.hadoop.ipc.Client$


Posts: 1,903
Kudos: 435
Solutions: 307
Registered: ‎07-31-2013

Re: when accessing CDH4 hdfs from CDH5 cluser nod

Please visit
see what the access matrix and point of running command is for CDH4 <->
CDH5 transfers via DistCp.