question JsonTreeReader changing time zones in NIFI in Support Questions

JsonTreeReader changing time zones in NIFI

Galih — Fri, 04 Aug 2023 03:12:48 GMT

I have record from kafka like this

{"datetime": "2023-08-03 10:46:50", "long": 106.8451, "lat": -6.2146, "city": "Jakarta", "status": "unknown", "nickname": "unknown"}

I use Convertrecord JsonTreeReader with Schema Access Strategy Infer Schema and to change Parquet before I stored in Hive

But in my table in Hive the datetime change

from 2023-08-03 10:46:50

to 2023-08-03 17:46:50

How to Handle that, I want to stay with same kafka record? I live in GMT+7

Re: JsonTreeReader changing time zones in NIFI

Galih — Sat, 05 Aug 2023 03:20:51 GMT

@VidyaSargur can you help me?

Re: JsonTreeReader changing time zones in NIFI

cotopaul — Mon, 07 Aug 2023 08:04:56 GMT

@Galih, as far as I know, your date will be converted to GMT per default. You could try using an UpdateRecord instead of an ConvertRecord and add a property name "/YOUR_DATE_COLUMN_NAME", having the value: format( /YOUR_DATE_COLUMN_NAME, "yyyy/MM/dd/HH", "THE_TIMEZONE_YOU_ARE_IN"). This is what I am using right now to avoid this exact behavior, especially when working with different formats (JSON to AVRO, AVRO to CSV, etc)

Re: JsonTreeReader changing time zones in NIFI

Galih — Mon, 07 Aug 2023 08:32:26 GMT

@cotopaul Like this? I'm new for converting

Re: JsonTreeReader changing time zones in NIFI

cotopaul — Mon, 07 Aug 2023 12:26:55 GMT

yes, you can try like that or like :
format( /datetime, "yyyy/MM/dd/HH", "Asia/Jakarta") - but you will need to switch Replacement Value strategy, from Literal Value to Record Path Value.