Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 08-03-2023 08:12 PM
I have record from kafka like this
{"datetime": "2023-08-03 10:46:50", "long": 106.8451, "lat": -6.2146, "city": "Jakarta", "status": "unknown", "nickname": "unknown"}
I use Convertrecord JsonTreeReader with Schema Access Strategy Infer Schema and to change Parquet before I stored in Hive
But in my table in Hive the datetime change
from 2023-08-03 10:46:50
to 2023-08-03 17:46:50
How to Handle that, I want to stay with same kafka record? I live in GMT+7
Created 08-07-2023 05:26 AM
yes, you can try like that or like :
format( /datetime, "yyyy/MM/dd/HH", "Asia/Jakarta") - but you will need to switch Replacement Value strategy, from Literal Value to Record Path Value.
Created 08-04-2023 08:20 PM
@VidyaSargur can you help me?
Created 08-07-2023 01:04 AM
@Galih, as far as I know, your date will be converted to GMT per default. You could try using an UpdateRecord instead of an ConvertRecord and add a property name "/YOUR_DATE_COLUMN_NAME", having the value: format( /YOUR_DATE_COLUMN_NAME, "yyyy/MM/dd/HH", "THE_TIMEZONE_YOU_ARE_IN"). This is what I am using right now to avoid this exact behavior, especially when working with different formats (JSON to AVRO, AVRO to CSV, etc)
Created 08-07-2023 01:32 AM
@cotopaul Like this? I'm new for converting
Created 08-07-2023 05:26 AM
yes, you can try like that or like :
format( /datetime, "yyyy/MM/dd/HH", "Asia/Jakarta") - but you will need to switch Replacement Value strategy, from Literal Value to Record Path Value.