Reply
New Contributor
Posts: 4
Registered: ‎03-28-2017

error processing READ_BLOCK operation

In my HDFS health summary and logs i am able to see the below error.

 

ERROR org.Apache.hadoop.hdfs.server.database.DataNode: abc@xyz.com:1004: DataXceiver error processing Roadblock operation..

 

I tried to rebalance but still i see the warnings in HDFS and also above error in /var/log/hadoop-hdfs.

 

I do saw a open jira https://issues.apache.org/jira/browse/HDFS-11346 but not sure if this is applicable to my issue or errors o notice.

 

Any help appreciated.

Posts: 642
Topics: 3
Kudos: 105
Solutions: 67
Registered: ‎08-16-2016

Re: error processing READ_BLOCK operation

Is it always the same host it is trying to read from? Can you provide the full stack trace? What do you have set for Maximum Number of Transfer Threads
dfs.datanode.max.xcievers, dfs.datanode.max.transfer.threads?
New Contributor
Posts: 4
Registered: ‎03-28-2017

Re: error processing READ_BLOCK operation

Yes itas always the same host thats reading..

I am not able to the see the values or entrire for dfs.datanode.max.xcievers, dfs.datanode.max.transfer.threads in the hdfs-site.xml

Posts: 642
Topics: 3
Kudos: 105
Solutions: 67
Registered: ‎08-16-2016

Re: error processing READ_BLOCK operation

What version of CDH are you running? How many total nodes do you have?

The same host causing this error would lead me to look for hardware issues with that host. Are the NICs dropping any packets, any error packets, etc.? If possible tests the disks using something like dd to ensure that it isn't one or more fault drives that are ready to fail but haven't just yet.
New Contributor
Posts: 4
Registered: ‎03-28-2017

Re: error processing READ_BLOCK operation

CDH version is 5.4.4.

There are just 18 nodes in the cluster.there are no hardware alerts if so by now the unix monitoring team would have raised the flag or ticket on it.

Announcements