Member since
08-09-2017
2
Posts
0
Kudos Received
0
Solutions
08-09-2017
10:09 AM
Okay. I monitored the logs and saw that nothing was really happening on the data node or the data directories on it. So I went ahead and aborted the configuration refresh. I also ended up restarting the entire cluster afterwards and, miraculously, everything is alright now. My Name Node is not reporting any issues, the data node configuration has come into effect and the name node is also able to see the data node now. I can run queries in Hive and Spark is also not complaining. `hdfs fsck / -files -blocks -locations > dfs-new-fsck-4.log` also reported the below findings, which tells me everything is good. Status: HEALTHY
Total size: 1892542036524 B (Total open files size: 134227921 B)
Total dirs: 745065
Total files: 2651007
Total symlinks: 0 (Files currently being written: 3)
Total blocks (validated): 2652077 (avg. block size 713607 B) (Total open file blocks (not validated): 4)
Minimally replicated blocks: 2652077 (100.0 %)
Over-replicated blocks: 11083 (0.41789886 %)
Under-replicated blocks: 1 (3.7706297E-5 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 2.994875
Corrupt blocks: 0
Missing replicas: 1 (1.2607865E-5 %)
Number of data-nodes: 4
Number of racks: 1
FSCK ended at Wed Aug 09 19:01:18 CEST 2017 in 43196 milliseconds
The filesystem under path '/' is HEALTHY Apologies for wasting everyone's time here.
... View more