Support Questions
Find answers, ask questions, and share your expertise

Hive not able to read ID column with data type int/big int

Highlighted

Hive not able to read ID column with data type int/big int

Expert Contributor

I have data in a flat file that my hive table is reading from. It reads rest of the data just fine. However, I have a column with values like

408574322

which is typically an id and suggested datatype of hue while creating the table is int. I tried changing that to bigint, double and varchar but yet it shows NULL for all the values in the column.

However if I change it to a string, it shows this with in the quotes i.e

"408574322".How do I fix this?Why is it not able to read this sort of id's anywhere in any of the tables?

6 REPLIES 6
Highlighted

Re: Hive not able to read ID column with data type int/big int

@Simran Kaur

Int does allow values between INT (4-byte signed integer, from -2,147,483,648 to 2,147,483,647). Refer to below link:

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types

I don't think it is related to data type precision issue. Share the logs under /var/log/hue for analysis.

Highlighted

Re: Hive not able to read ID column with data type int/big int

Guru

Can you share hiveserver logs ? also try casting and see if you are able to see the results that way.

Highlighted

Re: Hive not able to read ID column with data type int/big int

Expert Contributor

@srai: Which file in this folder would contain the relevant logs?

Highlighted

Re: Hive not able to read ID column with data type int/big int

Expert Contributor

Now, as I check further, it stored 0's for a lot of columns with data types int/smallint/float and double as well @srai

Highlighted

Re: Hive not able to read ID column with data type int/big int

@Simran Kaur

Can you check if the query returns 0 when query is run from hive cli or beeline?

If not, attach beeswax log, runcpserver log from /var/log/hue and hiveserver2.log from /var/log/hive.

Highlighted

Re: Hive not able to read ID column with data type int/big int

Guru

Have you already tried doing a create table as select @Simran Kaur, any luck with that ? Also, can you try this and share the output (make sure you have enough screen scroll buffer)

hive --hiveconf hive.root.logger=DEBUG,console