Reply
New Contributor
Posts: 1
Registered: ‎07-19-2017

Hivecontext is providing 1 extra count than the base hive table count

Hello Everyone,

 

when i am executing the query in hive as "select count(1) from table Table_Name" it is providing count as 100, But when i am executing same query from spark as "var RecordCount = hiveContext.sql("select count(1) from table Table_Name")" it is providing count as 101.

Please help me to resolve this issue.

 

Regards

Manoj

Posts: 642
Topics: 3
Kudos: 121
Solutions: 67
Registered: ‎08-16-2016

Re: Hivecontext is providing 1 extra count than the base hive table count

What version of CDH and Spark are you using?
Highlighted
Posts: 642
Topics: 3
Kudos: 121
Solutions: 67
Registered: ‎08-16-2016

Re: Hivecontext is providing 1 extra count than the base hive table count

I just tested with CDH 5.11.1 and Spark 1.6. The table I used is very simple and only has 5 records. Both systems returned the same and expected value.

This query doesn't look right thought "select count(1) from table Table_Name". It shouldn't have the 'table' keyword in it.

Note: if you include more than one valid table name in the query it will query the first table only and produce no errors.