Reply
New Contributor
Posts: 1
Registered: ‎07-26-2018

Timestamp issue when importing data

Hello,

 

I am trying to import data into kudu tables via Talend.

In Kudu, I have 2 columns that I am using to test how to store date values: one is one type BigInt and the other of type Timestamp.

In Talend, I convert the date value into long and it saves the data correctly

 

SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
Date date = format.parse(input_row.agencystartdate.toString());
long millis = date.getTime();
output_row.agencystartdatemillis = millis;

for example, "2007-09-04 00:00:00.0" gets saved as 1,188,849,600,000 (which when converted gived the date correctly)

 

When I save the same data in the the Kudu Timestamp column, the value is "14/01/1970 18:14:10" which is not good. All my dates are being saved in the year 1970.

 

 

How should I format the Date value to save it correctly in Kudu Timestamp?

Highlighted
Cloudera Employee
Posts: 19
Registered: ‎03-16-2017

Re: Timestamp issue when importing data

Hi,

 

Probably, the problem with those dates being back in 1970 comes from the fact that the value stored in the Kudu

UNIXTIME_MICROS column is interpreted as number of microseconds from the start of the Epoch?

 

Maybe, you can take a look into the Kudu Java client code to get some examples on working with timestamp columns: 

  https://github.com/apache/kudu/blob/master/java/kudu-client/src/test/java/org/apache/kudu/client/Tes...

  https://github.com/apache/kudu/blob/master/java/kudu-client/src/test/java/org/apache/kudu/client/Tes...

 

java.sql.Timestamp class might be helpful in various date/time conversions.

 

Hope that helps.

Announcements