Support Questions
Find answers, ask questions, and share your expertise

Impala connection question

Solved Go to solution
Highlighted

Impala connection question

Contributor

Hi,

 

I have a hadoop with one mater and two datanode, accessing hdfs can hit the master url, have impala on those two nodes as well, when connect through jdbc i cannot connect with the mater url, have to connect directly through individual node, is this by design or i missed something?

 


Thanks

Shannon

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Impala connection question

Champion
Yes, this is by design. The master roles for Impala preform functions for all Impala daemons, like caching metadata from HMS and HDFS block locations, maintaining a list of available Impala daemons, etc. But they do not manage connections. Each individual Impala daemon will managed the connections made to it and act as the coordinator for those connections.

For production, I recommend putting a load balancer in front of your Impala daemons to spread that connection load across all. Otherwise, having all users connect to a single one will exhaust the memory for that Impala daemon quickly. Another option I have seen is assigning blocks of Impala daemons to specific users groups.

View solution in original post

4 REPLIES 4
Highlighted

Re: Impala connection question

Champion
Yes, this is by design. The master roles for Impala preform functions for all Impala daemons, like caching metadata from HMS and HDFS block locations, maintaining a list of available Impala daemons, etc. But they do not manage connections. Each individual Impala daemon will managed the connections made to it and act as the coordinator for those connections.

For production, I recommend putting a load balancer in front of your Impala daemons to spread that connection load across all. Otherwise, having all users connect to a single one will exhaust the memory for that Impala daemon quickly. Another option I have seen is assigning blocks of Impala daemons to specific users groups.

View solution in original post

Highlighted

Re: Impala connection question

Contributor

Thanks.

Highlighted

Re: Impala connection question

Contributor

A related question, for running impala queries in HUE, i am hitting master, does it randomly hit one of the boxes?

Highlighted

Re: Impala connection question

Champion
Do you mean that HUE is installed on a master node? The HUE configs will contain an [impala] section and that will contain the Impala Daemon used by HUE. It will only connect to that ImpalaD.