Reading zero bytes issue in apache hadoop

[#1]

24/08/29 10:21:15 WARN DataStreamer: Exception for BP-942923949-172.20.0.202-1722967626672:blk_1074290600_560526
Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: java.io.EOFException: Unexpected EOF while trying to read response from server
Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:521)
Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1137)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: 24/08/29 10:27:30 WARN DataStreamer: DataStreamer Exception
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: java.net.SocketTimeoutException: 495000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/172.20.0.194:40396 remote=/172.20.0.194:9866]
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:163)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:158)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:116)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at java.io.DataOutputStream.write(DataOutputStream.java:107)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DFSPacket.writeTo(DFSPacket.java:193)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DataStreamer.sendPacket(DataStreamer.java:857)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:762)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: 24/08/29 10:27:30 WARN DataStreamer: Error Recovery for BP-942923949-172.20.0.202-1722967626672:blk_1074290600_560526 in pipeline [DatanodeInfoWithStorage[172.20.0.194:9866,DS-d8020c9c-24d1-462a-b707-0d219ce5848a,DISK], DatanodeInfoWithStorage[172.20.0.204:9866,DS-5a287a01-fda4-4ad0-a3e6-9d24cc6d8568,DISK], DatanodeInfoWithStorage[172.20.0.196:9866,DS-90f354a6-9531-49dd-97c4-af388f697c34,DISK]]: datanode 0(DatanodeInfoWithStorage[172.20.0.194:9866,DS-d8020c9c-24d1-462a-b707-0d219ce5848a,DISK]) is bad.
last 2 months we have had the issue please help me due to this our script got stuck and the lag has increased
Please help me

[#2]

Hi @HadoopCommunity 

This job is failing due to issues retrieving the block from datanodes. Please check the datanode logs during the issue timestamp to check the actual cause of the issue.

[#3]

Hi @HadoopCommunity 

Was your question answered? Please take some time to click on "Accept as Solution" -- If you find a reply useful, say thanks by clicking on the thumbs up button below this post.