question Re: Impala connection question in Archives of Support Questions (Read Only)

Impala connection question

ponypony — Fri, 16 Sep 2022 11:42:36 GMT

Hi,

I have a hadoop with one mater and two datanode, accessing hdfs can hit the master url, have impala on those two nodes as well, when connect through jdbc i cannot connect with the mater url, have to connect directly through individual node, is this by design or i missed something?

Thanks

Shannon

Re: Impala connection question

mbigelow — Tue, 06 Jun 2017 17:50:40 GMT

Yes, this is by design. The master roles for Impala preform functions for all Impala daemons, like caching metadata from HMS and HDFS block locations, maintaining a list of available Impala daemons, etc. But they do not manage connections. Each individual Impala daemon will managed the connections made to it and act as the coordinator for those connections.

For production, I recommend putting a load balancer in front of your Impala daemons to spread that connection load across all. Otherwise, having all users connect to a single one will exhaust the memory for that Impala daemon quickly. Another option I have seen is assigning blocks of Impala daemons to specific users groups.

Re: Impala connection question

ponypony — Tue, 06 Jun 2017 18:50:08 GMT

Thanks.

Re: Impala connection question

ponypony — Tue, 06 Jun 2017 18:57:04 GMT

A related question, for running impala queries in HUE, i am hitting master, does it randomly hit one of the boxes?

Re: Impala connection question

mbigelow — Tue, 06 Jun 2017 22:15:24 GMT

Do you mean that HUE is installed on a master node? The HUE configs will contain an [impala] section and that will contain the Impala Daemon used by HUE. It will only connect to that ImpalaD.