Member since
10-16-2024
11
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1321 | 10-16-2024 06:28 PM |
08-19-2025
01:48 AM
Hi, @allen_chu Your jstack shows many DataXceiver threads stuck in epollWait, meaning the DataNode is waiting on slow or stalled client/network I/O. Over time, this exhausts threads and makes the DataNode unresponsive. Please check network health and identify if certain clients (e.g., 172.18.x.x) are holding connections open. Review these configs in hdfs-site.xml: dfs.datanode.max.transfer.threads, dfs.datanode.socket.read.timeout, and dfs.datanode.socket.write.timeout to ensure proper limits and timeouts. Increasing max threads or lowering timeouts often helps. Also monitor for stuck jobs on the client side.
... View more
07-15-2025
02:22 AM
https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common/3.1.1.3.1.5.0-152
... View more
04-28-2025
07:05 AM
@Shelton Please read my previous answer carefully. None of the properties provided by you are in hbase codebase
... View more
12-19-2024
06:44 PM
@Shelton Thank you for your reply. This information is very helpful.
... View more
10-16-2024
06:28 PM
1 Kudo
Hi everyone, Thank you all for your responses. I am using Spark 3, and I’ve discovered that the issue is due to the improper configuration of the spark_shuffle settings in the yarn-site.xml file. Thanks again!
... View more