Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Can we fix the under Under replicated blocks while doing the copyToLocal

avatar
New Contributor

Hi Guys,

 

I am working on a backup task in DEV. I am doing the datanode backup to local. I have set the safemode is OFF and started the data backup. While in the middle of backup, I have noticed there is 1 under replicated blocks and 1 missing block. So i am planning to do the fix. So my doubt is, We can do the fix parallely ?

1 ACCEPTED SOLUTION

avatar
Champion

@Iron

 

In my opinion both are two different tasks, and it is upto you to do it in parallel

 

also the copyToLocal should work even with under replicated block as only one replication out of three (if you follow default 3 replication) will be copied to local

 

also not sure i am missing something here, but wondering why you are using copyToLocal option for backup instead of Cloudera Manager -> Backup (menu) -> Replication schedule -> Create schedule (pre-request: set the peer) option, it may reduce your work

View solution in original post

3 REPLIES 3

avatar
Champion

@Iron

 

In my opinion both are two different tasks, and it is upto you to do it in parallel

 

also the copyToLocal should work even with under replicated block as only one replication out of three (if you follow default 3 replication) will be copied to local

 

also not sure i am missing something here, but wondering why you are using copyToLocal option for backup instead of Cloudera Manager -> Backup (menu) -> Replication schedule -> Create schedule (pre-request: set the peer) option, it may reduce your work

avatar
New Contributor

Currently we dont have cloudera manager. so any other alternate method other than copyToLocal?. and also we have mandatory to enable the Safemode on for doing the copyTolocal? 

avatar
Champion

@Iron

 

I don't think it is mandatory to enable the safemode during copyToLocal. May be you can use safemode to make sure nobody is updating/deleting/inserting the data during the data copy.

 

I know the difficulties without cloudera manager/hortonworks, etc

 

Long back, i've used the below export/import method for Hive table backup, again this will export the data to HDFS and you still have to use copyTolocal. The advantage is, it will also take care of metadata

 

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ImportExport#LanguageManualImportExp...

 

You can use these options as temporary solution but once you start using cloudera manager or any other management tool, I would recommend to use the backup option that i've mentioned earlier