Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Impala Shell switches on Kerberized Cluster with HAProxy....how do I connect ?

avatar
Rising Star

Hi,

 

We have a CDH 5.9.0 cluster that has been kerberized....with Microsoft 2012 R2 Active Directory acting as the AD/LDAP and Kerberos domain and realm.

 

AD/LDAP "ldapsearch" commands work, GSSAPI works, "ktutil", "klist" and "kinit" all work for various users (including my "dreeves" user).

 

We have also setup the HAProxy load balancer for Impala (HAProxy is running on a non-CDH worker node via port 25003 and we have 4 x CDH worker nodes running impalad).

 

I can connect using the Hue GUI (and use both the Hive Query and Impala Query editors to run Hive HQL and Impala QL queries).

 

I can also connect via "impala-shell" command line using:

 

[dreeves@{obfuscated_fqdn_client_machine} ~]$  impala-shell

 

Starting Impala Shell without Kerberos authentication
Kerberos ticket found in the credentials cache, retrying the connection with a secure transport.
Error connecting: TTransportException, Could not connect to {obfuscated_fqdn_client_machine}:21000
***********************************************************************************

Welcome to the Impala shell.
(Impala Shell v2.7.0-cdh5.9.0 (4b4cf19) built on Fri Oct 21 01:07:22 PDT 2016)

Run the PROFILE command after a query has finished to see a comprehensive summary
of all the performance and diagnostic information that Impala gathered for that
query. Be warned, it can be very long!
***********************************************************************************
[Not connected] >  

 

[Not connected] >  connect {obfuscated_fqdn_haproxy_client_machine}:25003;
Connected to {obfuscated_fqdn_haproxy_client_machine}:25003
Server version: impalad version 2.7.0-cdh5.9.0 RELEASE (build 4b4cf1936bd6cdf34fda5e2f32827e7d60c07a9c)

 

[{obfuscated_fqdn_haproxy_client_machine}:25003] > show databases;

 

....

.... List of databases

.... my_dev

....

 

[{obfuscated_fqdn_haproxy_client_machine}:25003] > exit;

 

 

 

However, I am unable to connect using either of the following impala-shell commands:

 

 

1)  This command tries to use the client machine where I've installed HAProxy and port 25003:

 

[dreeves@{obfuscated_fqdn_client_machine} ~]$   impala-shell -l -u dreeves@CDH.{OBFUSCATED_REALM}.COM.AU --ssl --database=my_dev --impalad={obfuscated_fqdn_haproxy_client_machine}:25003;

 

Starting Impala Shell using LDAP-based authentication

 

SSL is enabled. Impala server certificates will NOT be verified (set --ca_cert to change)

 

LDAP password for dreeves@CDH.{OBFUSCATED_REALM}.COM.AU:    {my_obfuscated_LDAP_password}

 

Error connecting: TTransportException, Could not connect to {obfuscated_fqdn_haproxy_client_machine}:25003

 

Kerberos ticket found in the credentials cache, retrying the connection with a secure transport.

 

Error connecting: TTransportException, Could not connect to {obfuscated_fqdn_haproxy_client_machine}:25003
***********************************************************************************
Welcome to the Impala shell.
(Impala Shell v2.7.0-cdh5.9.0 (4b4cf19) built on Fri Oct 21 01:07:22 PDT 2016)

The HISTORY command lists all shell commands in chronological order.
***********************************************************************************
[Not connected] >

 

 

 

2)  This commands tries to use my LDAP user id and SSL to one of the machines where an impalad runs on port 21000:

 

[dreeves@{obfuscated_fqdn_client_machine} ~]$  impala-shell -l -u dreeves@CDH.{OBFUSCATED_REALM}.COM.AU --ssl --database=my_dev --impalad={obfuscated_fqdn_impalad_worker_machine}:21000;

 

Starting Impala Shell using LDAP-based authentication

 

SSL is enabled. Impala server certificates will NOT be verified (set --ca_cert to change)

 

LDAP password for dreeves@CDH.{OBFUSCATED_REALM}.COM.AU:      {my_obfuscated_LDAP_password}

 

Error connecting: TTransportException, Could not connect to {obfuscated_fqdn_impalad_worker_machine}:21000

 

Kerberos ticket found in the credentials cache, retrying the connection with a secure transport.

 

Error connecting: TTransportException, Could not connect to {obfuscated_fqdn_impalad_worker_machine}:21000

 

***********************************************************************************
Welcome to the Impala shell.
(Impala Shell v2.7.0-cdh5.9.0 (4b4cf19) built on Fri Oct 21 01:07:22 PDT 2016)

Want to know what version of Impala you're connected to? Run the VERSION command to
find out!
***********************************************************************************
[Not connected] >

 

 

 

Is someone able to confirm if I am entering correct "impala-shell" commands at 1) and 2) ?

 

If they are correct I can go away and look in the /var/log/impalad/ location for potential issues....

 

 

Thanks,

 

Damion.

1 REPLY 1

avatar
Champion
Kerberos and LDAP for Impala are different. MS AD provide both but configuring Hadoop and Impala for Kerberos uses that for auth and not LDAP. Based on what you said and the message from using the base command I'd say you are using Kerberos. Below is the command you need to use.

impala-shell -k -i FQDN --ssl