Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CDH 6.3.2 Hive has significant performance difference compared it on CDH 5.14

CDH 6.3.2 Hive has significant performance difference compared it on CDH 5.14

New Contributor

Hi,
We have a small table which only includes 100 records in Hive on CDH 6.3.2, do a select query via hive shell, it elapsed 27431.491 seconds (7.6 hours).
While the exact same operation only took 43.18 seconds on CDH 5.14.4.

 

These 2 CDH clusters all are with default settings, and they are 3 workers(datanode, nodemanager), and the Hive server is on the same host of namenode/resource manager.
Each node is 8 core 16G mem, I think this is sufficient for this simple testing.

 

Is there some change with CDH 6.3.2? Or do we need some extra config with Hive?(But many other tests all are worked well as expected.)
Very appreciate that some one can look into this issue, thanks a lot!

 

Reproduce step:
(1) Create a table like this via hive shell:
hive> desc testTable;
OK
recordid varchar(400)
innteger bigint
real double
string varchar(400)
daate date
tiime timestamp
datetime timestamp

 

(2) Fill some records like this one:
R1,38,28.5,lbxIminQQc,2020-04-20,10:49:10.0,2011-03-13 16:43:14.0

 

(3) In hive shell, execute the below statement, it will hung there:
SELECT
T0.recordID AS recordID ,
T0.innteger AS innteger ,
T0.real AS real ,
T0.string AS string ,
T0.daate AS daate ,
T0.tiime AS tiime ,
T0.datetime AS datetime ,
((((T0.innteger) * (10))) - (50)) AS `newfield1` ,
((CASE WHEN 2 = 0 THEN NULL ELSE (((((((T0.innteger) * (10))) - (50)))) / (2)) END)) AS `newfield2` ,
(((((((CASE WHEN 2 = 0 THEN NULL ELSE (((((((T0.innteger) * (10))) - (50)))) / (2)) END)))) + (10))) * (0.3)) AS `newfield3` ,
((((((((((CASE WHEN 2 = 0 THEN NULL ELSE (((((((T0.innteger) * (10))) - (50)))) / (2)) END)))) + (10))) * (0.3)))) + (15.5)) AS `newfield4` ,
(CASE WHEN (((((((((((((CASE WHEN 2 = 0 THEN NULL ELSE (((((((T0.innteger) * (10))) - (50)))) / (2)) END)))) + (10))) * (0.3)))) + (15.5)))) < (70.0)) THEN 'T' ELSE 'F' END) AS `newfield5`
FROM
testTable T0

 

Don't have an account?
Coming from Hortonworks? Activate your account here