Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created on
11-27-2017
04:45 AM
- last edited on
04-21-2026
06:45 AM
by
GrazittiAPI
Hi
I'm attemtping to import a csv file into a table into Hadoop.
The file has imported but the fields with a timestamp field have returned as null for all rows (no nulls in original file).
I've currently formatted the columns in the csv file as custom format yyyy-mm-dd hh:dd:ss
and set the field in the Hadoop table as a timestamp. Presumably the excel format isn't being recognised.
How can I resolve this?
Thanks
Created on 11-28-2017 11:27 AM - edited 11-28-2017 11:29 AM
You can create a table with a text column first, and then use to_timestamp to convert it into a table with a timestamp column.
create table timestamp_table as select column_1, column_2, to_timestamp(column_3, "dd/MM/yyyy HH:mm") from text_table;
For documentations on timestamp-related functions, please visit https://impala.apache.org/docs/build/html/topics/impala_datetime_functions.html
Created on 11-27-2017 02:29 PM - edited 11-27-2017 02:39 PM
The standard timestamp format is"yyyy-MM-dd HH:mm:ss[.SSSSSS]". Is there any warning printed when selecting from this table? Could you share a sample row in the text file?
Created 11-28-2017 06:04 AM
Example of the field is
27/01/2016 11:25:00
None of the fields have any second values. They're all 00.
Within the file it's formatted as dd/mm/yyyy hh:mm.
I've tried changing it to yyyy-mm-dd hh:mm:ss but when I do that, save and re-open it's reverted to the original format.
I can upload into a text field (and it appears as 27/01/2016 11:25) but then if I cast the field as a timestamp it then returns null values.
Created on 11-28-2017 11:27 AM - edited 11-28-2017 11:29 AM
You can create a table with a text column first, and then use to_timestamp to convert it into a table with a timestamp column.
create table timestamp_table as select column_1, column_2, to_timestamp(column_3, "dd/MM/yyyy HH:mm") from text_table;
For documentations on timestamp-related functions, please visit https://impala.apache.org/docs/build/html/topics/impala_datetime_functions.html
Created 12-01-2017 03:52 AM
Thanks, this function worked