Support Questions
Find answers, ask questions, and share your expertise

Wrong Hive table schema from Livy interpreters

New Contributor

Hi everybody,

I've got an external Hive table which is stored as ORC files. When I request and describe it from livy.sql and livy.pyspark interpreters, schema is wrong. Indeed, it returns the right column types but got the wrong column names. Each column name is _colXXX where XXX is an integer.

Conversely, when I request the table from the Hive engine the column names and the schema are ok.

I checked ORC files, some of them contain wrong column names written _colXXX but the majority of them embark the right schema. How to explain that ?

Should I conclude that Hive engine read schemas on his metastore database and Livy interpreters on ORC files ? How Livy interpreters can read the right schema ?

Thanks a lot in advance guys !

1 REPLY 1

New Contributor

A little up 🙂