Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS data blocks are recovered and cluster status = HEALTHY but file sequences are missing for specific table.

HDFS data blocks are recovered and cluster status = HEALTHY but file sequences are missing for specific table.

Contributor

logs-mx.txtKindly find the attached log file & details:log-status.txt

HDFS file is not in proper sequence. HDFS makes the file starting with 0 and it goes in a sequence manner as we believed.

For a specific table and its corresponding HDFS location,

we are not seeing the block 0_0 and 7_0 as per the sequence. These two blocks are creating the missing data which, goes with the below dfsadmin report.

please let us know why the mentioned file sequence in the table is missing

2 REPLIES 2

Re: HDFS data blocks are recovered and cluster status = HEALTHY but file sequences are missing for specific table.

Mentor

@hardik desai

What is the replication factor set hdfs-site.xml:

<property> 
<name>dfs.replication<name> 
<value>3<value> 
<description>Block Replication<description> 
<property>

Fix Under-replicated blocks in HDFS

su - <$hdfs_user> 
$ hdfs fsck / | grep 'Under replicated' | awk -F':' '{print $1}' >> /tmp/under_replicated_files  
$ for hdfsfile in `cat /tmp/under_replicated_files`; do echo "Fixing $hdfsfile :" ;  hadoop fs -setrep 3 $hdfsfile; done

Then rerun dfsadmin report

Re: HDFS data blocks are recovered and cluster status = HEALTHY but file sequences are missing for specific table.

Contributor

@Geoffrey Shelton, Thanks for your comment. We have already tried that option and it didn't help us.

Don't have an account?
Coming from Hortonworks? Activate your account here