I am facing an issue with one of our Impala Daemon which continuously has error related to “The health test result for IMPALAD_FRONTEND_CONNECTIONS has become bad: There are 0 (Beeswax pool) 123 (Hive Server 2 pool) active client connections, each pool has a configured maximum of 128”
Have checked CM -> Impala -> Charts Library -> check "Active Frontend API connection" this shows too many Hive Server connections
Also checked server's network side and run netstat -an | grep ESTABLISHED | grep 25000 but I see only 2 active connections.
[root@nucleus ~]$ netstat -an | grep ESTABLISHED | grep 25000 tcp 0 0 10.20.20.41:25000 10.20.20.41:49728 ESTABLISHED
tcp 0 0 10.20.20.41:25000 10.20.20.41:49732 ESTABLISHED
Any assistance / guidance in resolving this issue is appericated.
It shows the number of impalad connections are hitting the limit.
You can check on these below points:
1. you can increase the value of "Impala Daemon Max Client Connections" property in the impala configs if really more number of connections are coming.
2. If you think the connections are less but the opened connections are not closing properly and causing the issue you can check the timeouts has been set properly in the Hue safety-valve advance configs.