Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Can we fix the under Under replicated blocks while doing the copyToLocal

SOLVED Go to solution

Can we fix the under Under replicated blocks while doing the copyToLocal

New Contributor

Hi Guys,

 

I am working on a backup task in DEV. I am doing the datanode backup to local. I have set the safemode is OFF and started the data backup. While in the middle of backup, I have noticed there is 1 under replicated blocks and 1 missing block. So i am planning to do the fix. So my doubt is, We can do the fix parallely ?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Can we fix the under Under replicated blocks while doing the copyToLocal

Champion

@Iron

 

In my opinion both are two different tasks, and it is upto you to do it in parallel

 

also the copyToLocal should work even with under replicated block as only one replication out of three (if you follow default 3 replication) will be copied to local

 

also not sure i am missing something here, but wondering why you are using copyToLocal option for backup instead of Cloudera Manager -> Backup (menu) -> Replication schedule -> Create schedule (pre-request: set the peer) option, it may reduce your work

3 REPLIES 3

Re: Can we fix the under Under replicated blocks while doing the copyToLocal

Champion

@Iron

 

In my opinion both are two different tasks, and it is upto you to do it in parallel

 

also the copyToLocal should work even with under replicated block as only one replication out of three (if you follow default 3 replication) will be copied to local

 

also not sure i am missing something here, but wondering why you are using copyToLocal option for backup instead of Cloudera Manager -> Backup (menu) -> Replication schedule -> Create schedule (pre-request: set the peer) option, it may reduce your work

Re: Can we fix the under Under replicated blocks while doing the copyToLocal

New Contributor

Currently we dont have cloudera manager. so any other alternate method other than copyToLocal?. and also we have mandatory to enable the Safemode on for doing the copyTolocal? 

Re: Can we fix the under Under replicated blocks while doing the copyToLocal

Champion

@Iron

 

I don't think it is mandatory to enable the safemode during copyToLocal. May be you can use safemode to make sure nobody is updating/deleting/inserting the data during the data copy.

 

I know the difficulties without cloudera manager/hortonworks, etc

 

Long back, i've used the below export/import method for Hive table backup, again this will export the data to HDFS and you still have to use copyTolocal. The advantage is, it will also take care of metadata

 

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ImportExport#LanguageManualImportExp...

 

You can use these options as temporary solution but once you start using cloudera manager or any other management tool, I would recommend to use the backup option that i've mentioned earlier