I've got an external Hive table which is stored as ORC files. When I request and describe it from livy.sql and livy.pyspark interpreters, schema is wrong. Indeed, it returns the right column types but got the wrong column names. Each column name is _colXXX where XXX is an integer.
Conversely, when I request the table from the Hive engine the column names and the schema are ok.
I checked ORC files, some of them contain wrong column names written _colXXX but the majority of them embark the right schema. How to explain that ?
Should I conclude that Hive engine read schemas on his metastore database and Livy interpreters on ORC files ? How Livy interpreters can read the right schema ?
Thanks a lot in advance guys !