Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Impala connection question

avatar
Contributor

Hi,

 

I have a hadoop with one mater and two datanode, accessing hdfs can hit the master url, have impala on those two nodes as well, when connect through jdbc i cannot connect with the mater url, have to connect directly through individual node, is this by design or i missed something?

 


Thanks

Shannon

1 ACCEPTED SOLUTION

avatar
Champion
Yes, this is by design. The master roles for Impala preform functions for all Impala daemons, like caching metadata from HMS and HDFS block locations, maintaining a list of available Impala daemons, etc. But they do not manage connections. Each individual Impala daemon will managed the connections made to it and act as the coordinator for those connections.

For production, I recommend putting a load balancer in front of your Impala daemons to spread that connection load across all. Otherwise, having all users connect to a single one will exhaust the memory for that Impala daemon quickly. Another option I have seen is assigning blocks of Impala daemons to specific users groups.

View solution in original post

4 REPLIES 4

avatar
Champion
Yes, this is by design. The master roles for Impala preform functions for all Impala daemons, like caching metadata from HMS and HDFS block locations, maintaining a list of available Impala daemons, etc. But they do not manage connections. Each individual Impala daemon will managed the connections made to it and act as the coordinator for those connections.

For production, I recommend putting a load balancer in front of your Impala daemons to spread that connection load across all. Otherwise, having all users connect to a single one will exhaust the memory for that Impala daemon quickly. Another option I have seen is assigning blocks of Impala daemons to specific users groups.

avatar
Contributor

Thanks.

avatar
Contributor

A related question, for running impala queries in HUE, i am hitting master, does it randomly hit one of the boxes?

avatar
Champion
Do you mean that HUE is installed on a master node? The HUE configs will contain an [impala] section and that will contain the Impala Daemon used by HUE. It will only connect to that ImpalaD.