Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Reading zero bytes issue in apache hadoop

avatar

24/08/29 10:21:15 WARN DataStreamer: Exception for BP-942923949-172.20.0.202-1722967626672:blk_1074290600_560526
Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: java.io.EOFException: Unexpected EOF while trying to read response from server
Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:521)
Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1137)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: 24/08/29 10:27:30 WARN DataStreamer: DataStreamer Exception
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: java.net.SocketTimeoutException: 495000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/172.20.0.194:40396 remote=/172.20.0.194:9866]
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:163)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:158)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:116)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at java.io.DataOutputStream.write(DataOutputStream.java:107)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DFSPacket.writeTo(DFSPacket.java:193)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DataStreamer.sendPacket(DataStreamer.java:857)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:762)
Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: 24/08/29 10:27:30 WARN DataStreamer: Error Recovery for BP-942923949-172.20.0.202-1722967626672:blk_1074290600_560526 in pipeline [DatanodeInfoWithStorage[172.20.0.194:9866,DS-d8020c9c-24d1-462a-b707-0d219ce5848a,DISK], DatanodeInfoWithStorage[172.20.0.204:9866,DS-5a287a01-fda4-4ad0-a3e6-9d24cc6d8568,DISK], DatanodeInfoWithStorage[172.20.0.196:9866,DS-90f354a6-9531-49dd-97c4-af388f697c34,DISK]]: datanode 0(DatanodeInfoWithStorage[172.20.0.194:9866,DS-d8020c9c-24d1-462a-b707-0d219ce5848a,DISK]) is bad.
last 2 months we have had the issue please help me due to this our script got stuck and the lag has increased
Please help me

 

2 REPLIES 2

avatar
Expert Contributor

Hi @HadoopCommunity 

This job is failing due to issues retrieving the block from datanodes. Please check the datanode logs during the issue timestamp to check the actual cause of the issue.

avatar
Expert Contributor

Hi @HadoopCommunity 

Was your question answered? Please take some time to click on "Accept as Solution" -- If you find a reply useful, say thanks by clicking on the thumbs up button below this post.