Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Falcon Mirror issue

Solved Go to solution

Falcon Mirror issue

Rising Star

I am facing checksum mismatch while copying data between two cluster.

Caused by: java.io.IOException: Check-sum mismatch. Also is there a way to ignore checksum through falcon UI? . Thanks in advance.

Check-sum mismatch between hdfs://test1:8020/raw/ech/incr/row_bn... and hdfs://test2:8020/raw/ech/.distcp.tmp.attempt_1471351807373_0.... Source and target differ in block-size.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Falcon Mirror issue

Contributor

@nyadav

It appears that block size is different in your two clusters . You can set flags -preserveBlockSize or -skipChecksum as below .

1. Suspend all Falcon jobs . 2. Modify template for falcon mirroring at /usr/hdp/current/falcon-server/data-mirroring/workflows/hdfs-replication-workflow.xml .Add the following argument at the end in argument list to preserve the block size . <arg>-preserveBlockSize</arg> <arg>true</arg> 3. Restart Falcon through Ambari. 4. Resubmit the job and verify if the HDFS mirror job is working fine now.

This will set this property for all mirror jobs.

2 REPLIES 2

Re: Falcon Mirror issue

Contributor

@nyadav

It appears that block size is different in your two clusters . You can set flags -preserveBlockSize or -skipChecksum as below .

1. Suspend all Falcon jobs . 2. Modify template for falcon mirroring at /usr/hdp/current/falcon-server/data-mirroring/workflows/hdfs-replication-workflow.xml .Add the following argument at the end in argument list to preserve the block size . <arg>-preserveBlockSize</arg> <arg>true</arg> 3. Restart Falcon through Ambari. 4. Resubmit the job and verify if the HDFS mirror job is working fine now.

This will set this property for all mirror jobs.

Re: Falcon Mirror issue

Rising Star

Thanks @prsingh, able to run the job after the change

Don't have an account?
Coming from Hortonworks? Activate your account here