Member since
12-30-2015
4
Posts
0
Kudos Received
0
Solutions
04-22-2016
09:40 AM
Actually it is datanode doing it. I guess I'll ask more about it as an HDFS topic. Thanks!
... View more
04-21-2016
04:10 PM
Also we didn't have fs.du.interval setting in our config so by default it should be 600000 ms but we are seeing it much more often than that.
... View more
04-21-2016
03:16 PM
Hi there, We are doing some testing on a very small cluster and we were experiencing some extra load by du command. It is affecting our testing results significantly and we are bypassing it by creating a symbolic link of du to a df command. Our testing steps: 1. on all nodes: echo 1 > /proc/sys/vm/drop_caches 2. run scripts Can anyone has a detailed explaination of how the du command gets triggered by impala( we assume it's something related with vfs caching). Is there a config or a better way to make it not doing du? Thanks a lot! Let me know if you need more information. 🙂
... View more
Labels:
- Labels:
-
Apache Impala