- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Issue with Time Shift During Data Migration from MSSQL to PostgreSQL Using NiFi
- Labels:
-
Apache NiFi
Created 08-22-2024 04:03 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am currently migrating data from MSSQL to PostgreSQL. While fetching the data from MSSQL, I use the following query to ensure the time is in UTC:
SELECT "time column" AT TIME ZONE 'UTC' AS "time column"
This gives me the correct result in UTC. However, when I ingest the data into PostgreSQL using NiFi's PutDatabaseprocessor, I notice that the time is being increased by 5 hours. The destination column in PostgreSQL is of type timestamp with time zone.
when I manually run an INSERT command in PostgreSQL with the same time value, the time remains unchanged. The time shift only occurs during the NiFi ingestion process. Any ideas on what could be causing this discrepancy?
Created 03-11-2025 04:16 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Ytch
Can you try this solution that has been accepted in this forum
MSSQL to PostgreSQL
happy hadooping
Created 03-18-2025 07:47 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The issue may be due to NiFi's PutDatabaseProcessor applying a local time zone conversion during data ingestion, causing the 5-hour shift. To fix this, ensure NiFi is explicitly set to handle UTC time zones for both reading and writing. Additionally, using ODBC for PostgreSQL could help by providing better control over time zone handling during the data transfer, ensuring consistency between MSSQL and PostgreSQL.
