Member since
08-20-2015
3
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2581 | 10-09-2015 05:18 AM |
10-09-2015
05:18 AM
After some more research we found that it wasn't a zombie spark job that was causing the resource usage. It was the HDFS blockscanner. Apparently the default configuration changed with the upgrade and it started running right after we restarted upon upgrading. We had never seen it running before and hence the mistery.
... View more
08-24-2015
06:49 AM
Nevermind, it was a zombie spark job.
... View more
08-20-2015
08:54 AM
Earlier this week I upgraded to CDH 5.4.5 from 5.4.4. Since then I can see that the datanode process is constantly reading 2M/s per drive in each host (with a number of writes maybe one order of magnitude smaller), but there is no corresponding HDFS I/O activity (just the usual activity, ~40k/s) It seems as if nobody is actually reading anything, but the HDFS process is doing something on its own. Any ideas about what could be causing this? How could I find out / diagnose what's happening? Thanks!
... View more
Labels: