Hello, Need some assistance / guidance on how we can reduce Non-HDFS Space. We see Non-HDFS Space of around 270 used, as we are facing space crunch, we would explore possibilities for getting non-hdfs space reduced. I have cleared all Yarn logs for the applications which were killed/ failed etc (our /data mountpoint houses dfs, yarn, kudu, impala), yet this does not solve our issue. Any assistance / guidance is much appreciated.   Thanks  Amn ... View more

Hello, We are getting alerts for Block Count on one of our data nodes as it has crossed the threshold of 10000. Since HDFS balancer did not fix the issue, the next thing I turned my focus to see if we are hitting small files issue. I was trying to put up a report via terminal script ( hdfs dfs -ls -R /tmp |grep ^- |awk '{if ($5 < 134217728) print $5, $8;}'| head -5 | column –t) but when I compare the result from the script output vs HDFS Report from Cloudera Manager I see a difference in the size of the same file.   Could anyone provide any guidance / assistance on this, or am I doing something wrong.   Thanks Amn ... View more

Hello, I need to increase the query timeout (backend_client_rpc_timeout_ms) from current 5 minutes to 30 minutes. Could anyone guide me from where in Impala Configurations I can do this change. I did some checking but cannot find any related, appreciate any assistance / guidance.   Thanks  Amn ... View more

Hello, We are seeing concerning alert on one of our data node related to File Descriptor (Concerning: Open file descriptors: 16,410. File descriptor limit: 32,768. Percentage in use: 50.08%. Warning threshold: 50.00%.)   Would appreciate any help/ guidance to fix this before it goes out of hand.   [user1@myserver ~]$ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 1030544 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 4096 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited   [user1@myserver ~]$ cat /proc/sys/fs/file-max 26161091 [user1@myserver ~]$ cat /proc/sys/fs/file-nr 80400 0 26161091   Thanks  Amn ... View more