I have an ORC table tbl1 having timestamp column. I inserted these values: insert into tbl1 values ('0001-01-01 00:00:20.0'); insert into tbl1 values ('9999-01-01 00:00:20.0'); I created another ORC table tbl2 having timestamp column. Then I tried: insert into tbl2 select * from tbl1; Data in tbl2 is corrupted: 1754-08-28 22:44:01.128654848 1815-03-31 05:56:28.066277376 Am I using out of range values? Is there any specific range for timestamp? This problem is only for ORC file format. For other file formats like textfile , avro , parquet , etc it's working fine. ... View more

Spark Version : 1.6.1 I have a text table in hive having `timestamp` datatype with nanoseconds precision. Hive Table Schema: c_timestamp timestamp Hive Table data: 00:00:00.000000001 23:59:59.999999999 But as per the docs, from Spark 1.5 Timestamps are now stored at a precision of 1us, rather than 1ns Sample code: SparkConf conf = new SparkConf(true).setMaster("yarn-cluster").setAppName("SAMPLE_APP"); SparkContext sc = new SparkContext(conf); HiveContext hc = new HiveContext(sc); DataFrame df = hc.table("testdb.tbl1"); Data is truncated to microseconds. Is there any way to use nanoseconds here? ... View more