Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Issue with Time Shift During Data Migration from MSSQL to PostgreSQL Using NiFi

avatar
Explorer

I am currently migrating data from MSSQL to PostgreSQL. While fetching the data from MSSQL, I use the following query to ensure the time is in UTC:

 

SELECT "time column" AT TIME ZONE 'UTC' AS "time column"

 

 This gives me the correct result in UTC. However, when I ingest the data into PostgreSQL using NiFi's PutDatabaseprocessor, I notice that the time is being increased by 5 hours. The destination column in PostgreSQL is of type timestamp with time zone.

when I manually run an INSERT command in PostgreSQL with the same time value, the time remains unchanged. The time shift only occurs during the NiFi ingestion process. Any ideas on what could be causing this discrepancy?

2 REPLIES 2

avatar
Master Mentor

@Ytch 
Can you try this solution that has been accepted in this forum
MSSQL to PostgreSQL 

happy hadooping

avatar
New Member

The issue may be due to NiFi's PutDatabaseProcessor applying a local time zone conversion during data ingestion, causing the 5-hour shift. To fix this, ensure NiFi is explicitly set to handle UTC time zones for both reading and writing. Additionally, using ODBC for PostgreSQL could help by providing better control over time zone handling during the data transfer, ensuring consistency between MSSQL and PostgreSQL.