I have a DataFrame with integer values but I store it as a StringType as I need to keep leading zeros. Something like following (sample):
I save this table as:
It's all fine up until this step.
The problem starts when I try to read it back. I use the following code to read it back (notice the base folder): sqlContext.read.format("orc").load("sometable")
What I get back is: field (it is blank at best and sometimes it is littered with junk) Spark Version Used: 1.6.2
... View more