Skip to main content
wert_1311
Gold ⭐️
February 11, 2021
Solved

Error while executing hdfs fsck command

Hello,

I am trying to execute hdfs fsck command and am getting below error:

 

[root@server1 root]# hdfs fsck / > /home/test/fsck_output_2-11-21
Connecting to namenode via http://server1.com:50070/fsck?ugi=hdfs&path=%2F
Exception in thread "main" java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
at org.apache.hadoop.hdfs.tools.DFSck.doWork(DFSck.java:363)
at org.apache.hadoop.hdfs.tools.DFSck.access$000(DFSck.java:72)
at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:161)
at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:158)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.hadoop.hdfs.tools.DFSck.run(DFSck.java:157)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.hdfs.tools.DFSck.main(DFSck.java:406)

 

Thanks

wert

GangWar
GangWarAnswer
Platinum ⭐️⭐️
February 11, 2021

@wert_1311 This could be a known issue with HDFS. 

 

The issue with fsck command was introduced in HDFS-2538 when fsck doesn't show dots by default. fsck uses http port, 9871, which has the 60 seconds timeout hardcoded. The issue is observed when the command executed for more than the configured timeout. 

Use -showprogress parameter that can help you. 


# hdfs fsck / -showprogress

Cheers! Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.

4 replies

GangWar
Platinum ⭐️⭐️
GangWarAnswer
Platinum ⭐️⭐️
February 11, 2021

@wert_1311 This could be a known issue with HDFS. 

 

The issue with fsck command was introduced in HDFS-2538 when fsck doesn't show dots by default. fsck uses http port, 9871, which has the 60 seconds timeout hardcoded. The issue is observed when the command executed for more than the configured timeout. 

Use -showprogress parameter that can help you. 


# hdfs fsck / -showprogress

Translate
Cheers! Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
wert_1311
wert_1311Author
Gold ⭐️
February 15, 2021

@GangWar  Thanks !!! that works 

Translate