Reply
Highlighted
ojo
New Contributor
Posts: 3
Registered: ‎06-25-2014

Impala : query running slow

Hi,
 
I am using Cloudera4.5 (3 nodes) cluster for Hadoop and Impala1.1 for execution of queries.
My namenode is having 40GB RAMand each datanodes is having 16GB RAM.
 
Size of my tables is as follows :
crmd_link : (No of Rows : 11074) (Size: 832.7KB)
crmd_partner : (No of Rows : 15360) (Size: 2.60MB)
but000 : (No of Rows : 82) (Size: 13.7KB)
 
Response time for below query is : 4.89s

 

SELECT if(XUBNAME = 'AX4787',concat(NAME_FIRST," ", NAME_LAST), 'header names') Requestor FROM (SELECT PARTNER_NO FROM sap_archive.crmd_partner INNER JOIN (SELECT GUID_SET FROM sap_archive.crmd_link WHERE GUID_HI = '4B3EF47BB4F70DF5E10000009B7DA717' AND OBJTYPE_SET ='07') link  ON GUID = link.GUID_SET AND PARTNER_FCT = '00000016') partner INNER JOIN sap_archive.but000 ON partner.PARTNER_NO = PARTNER_GUID LIMIT 1;

 For checking Response time, i have break this query as follows:

 

SELECT GUID_SET FROM sap_archive.crmd_link WHERE GUID_HI = '4B3EF47BB4F70DF5E10000009B7DA717' AND OBJTYPE_SET ='07'

Response time : 1.87s


SELECT PARTNER_NO FROM sap_archive.crmd_partner INNER JOIN (SELECT GUID_SET FROM sap_archive.crmd_link WHERE GUID_HI = '4B3EF47BB4F70DF5E10000009B7DA717' AND OBJTYPE_SET ='07') link  ON GUID = link.GUID_SET AND PARTNER_FCT = '00000016';

Response time : 3.73s

 I have tried to configure certain parameters for performance :

    Changed Impala Daemon Log Buffer Level to 0 .
    Changed memory limit used in /etc/default/impala to 95%.

After making above changes, i have restarted Impala Daemons
 
I have attached link to view output, when i am executing profile command for same(due to size limit of message i am not able to paste full output)

 

If you require any logs, Please let me know.
 
Please help me out
 
Thanks in Advance,
Ankita Singla

Posts: 1,885
Kudos: 424
Solutions: 299
Registered: ‎07-31-2013

Re: Impala : query running slow