question Re: PutDatabaseRecord or JsonTreeReader changing time zones? in Support Questions

PutDatabaseRecord or JsonTreeReader changing time zones?

Tokolosk — Mon, 19 Jul 2021 15:08:37 GMT

I live in a GMT +2 timezone but we collect all our data as GMT.

My PutDatabaseRecord processor receives JSON with one field being the timestamp, e.g.:

2021-07-19 14:52:39

If I look at the flowfile just before the PutDatabaseRecord processor this timestamp is correct. However it gets inserted into the database as 2021-07-19 16:52:39

I can't find any documentation on how to prevent this in 1.14.0.

Any idea what to do? Currently I'm subtracting an additional 2 hours to make up for this but that's not a good permanent fix.

Edit:

Is this perhaps related to this issue? @mburgess : if not and this is a bug then I can give more info.

https://issues.apache.org/jira/browse/NIFI-8768?jql=text%20~%20%22time%20zone%20JsonTreeReader%22

Edit 2: this worked previously in 1.12.2

Re: PutDatabaseRecord or JsonTreeReader changing time zones?

GabrielNifi — Mon, 19 Jul 2021 18:20:15 GMT

Nifi 1.14 PutDatabaseRecord DateTime Bug: writes using local timezone (adding or removing hours from DateTime column)

Re: PutDatabaseRecord or JsonTreeReader changing time zones?

GabrielNifi — Mon, 19 Jul 2021 18:42:42 GMT

Do not use string. "Use Avro Logical Types: TRUE". So, try converting to Avro before PutDatabaseRecord.

Re: PutDatabaseRecord or JsonTreeReader changing time zones?

Tokolosk — Tue, 20 Jul 2021 06:10:56 GMT

My data source is not QueryDatabaseTable, it is json received from another source. Do I need to convert JSON to AVRO now before using PutDatabaseRecord?

Also, if I do add a new ConvertRecord processor for every PutDatabaseRecord I have, these do not have the "Use Avro Logical Types: TRUE" option either.

Re: PutDatabaseRecord or JsonTreeReader changing time zones?

Tokolosk — Wed, 21 Jul 2021 06:11:29 GMT

Is there any way to reach the Devs to know if this is intentional or not?

I need to upgrade a bunch of servers from 1.12.2 to 1.14.0, but I also need to know if we first have to add a bunch of processors to deal with this timezone change from PutDatabaseRecord.

Re: PutDatabaseRecord or JsonTreeReader changing time zones?

DennisJaheruddi — Thu, 22 Jul 2021 14:47:10 GMT

I have seen a similar issue a while ago but that one got fixed, the best way to reach the experts to check if there is a known fix is to go to the Cloudera Support Portal, once you log in you should be able to log a support ticket.

Re: PutDatabaseRecord or JsonTreeReader changing time zones?

Tokolosk — Fri, 30 Jul 2021 09:07:20 GMT

Hi @DennisJaheruddi

Unfortunately seems I cannot access the support ticket area - not enough privileges.

All I need to know if this is a bug (in which case I stay with 1.12.1) or if this is a new "feature" in which case I need to update all my flows when upgrading to 1.14.0.

Re: PutDatabaseRecord or JsonTreeReader changing time zones?

DennisJaheruddi — Tue, 03 Aug 2021 21:00:37 GMT

I am honestly not sure, timezones are one of the most tricky topics so I cannot easily determine if there was a mistake that got fixed, or whether a new one is introduced.

If you have problems accessing the support portal please reach out to the cloudera contact person for your company, best to fix your support account before you try to log a support case during a production down situation 😉

Re: PutDatabaseRecord or JsonTreeReader changing time zones?

Tokolosk — Wed, 04 Aug 2021 06:12:02 GMT

Hi @DennisJaheruddi

I managed to speak to some devs on Slack and we found a related issue, posted below.

https://issues.apache.org/jira/projects/NIFI/issues/NIFI-8749

Hopefully it gets fixed soon so that we can upgrade to 1.14.0 +