Support Questions
Find answers, ask questions, and share your expertise

Timestamp issue when importing data

New Contributor

Hello,

 

I am trying to import data into kudu tables via Talend.

In Kudu, I have 2 columns that I am using to test how to store date values: one is one type BigInt and the other of type Timestamp.

In Talend, I convert the date value into long and it saves the data correctly

 

SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
Date date = format.parse(input_row.agencystartdate.toString());
long millis = date.getTime();
output_row.agencystartdatemillis = millis;

for example, "2007-09-04 00:00:00.0" gets saved as 1,188,849,600,000 (which when converted gived the date correctly)

 

When I save the same data in the the Kudu Timestamp column, the value is "14/01/1970 18:14:10" which is not good. All my dates are being saved in the year 1970.

 

 

How should I format the Date value to save it correctly in Kudu Timestamp?

2 REPLIES 2

Contributor

Hi,

 

Probably, the problem with those dates being back in 1970 comes from the fact that the value stored in the Kudu

UNIXTIME_MICROS column is interpreted as number of microseconds from the start of the Epoch?

 

Maybe, you can take a look into the Kudu Java client code to get some examples on working with timestamp columns: 

  https://github.com/apache/kudu/blob/master/java/kudu-client/src/test/java/org/apache/kudu/client/Tes...

  https://github.com/apache/kudu/blob/master/java/kudu-client/src/test/java/org/apache/kudu/client/Tes...

 

java.sql.Timestamp class might be helpful in various date/time conversions.

 

Hope that helps.

New Contributor

Hello,

For anyone who has this problem.

What I have observed is that in java/Talend, getTime() gives you a value in milliseconds.

In Kudu, it expects the value to be in microseconds.

So a factor of 1000.

In Talend or java, after I get my date.getTime() value, I multiple it by 1000.

Then in Kudu I get the correct date.

I do not know if this is correct for everyone, but for me I get the desired result.

 

; ;