Member since
01-31-2022
2
Posts
0
Kudos Received
0
Solutions
02-08-2022
05:16 AM
hello Yes i tested without HWC and i'm getting a correct date value. Yes the HWC is not required for external table, but as we are creating a common lib for all hive tables, we prefered use HWC. Here are the steps to reproduce the problem: 1- i'm creating a csv file file.csv with those values: a;1 b;2 c;3 that i save in this path /mypath/db/test/dt=2021-02-10. 2- create an external database : CREATE EXTERNAL TABLE db.test ( col1 string, col2 string) PARTITIONED BY (dt date) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\073' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/mypath/db/test'; 3- start a spark shell with HWC import com.hortonworks.hwc.HiveWarehouseSession val hive=HiveWarehouseSession.session(spark).build() hive.table("db.test").select("dt").distinct().show() +--------------+ | dt| +--------------+ |2021-02-09| +--------------+
... View more