question Re: Investigation of Future Timestamps Generated in NiFi Due to Server Timezone Change in Support Questions

Investigation of Future Timestamps Generated in NiFi Due to Server Timezone Change

NaveenSagar — Thu, 06 Mar 2025 06:27:12 GMT

We recently observed an unusual scenario in NiFi where timestamps like '20251229043001', '20251230043001', and '20251231043001' were generated for December 29, 2024, December 30, 2024, and December 31, 2024. These timestamps correspond to future dates, which seems abnormal. I suspect that this issue may have occurred due to a change in the timezone settings of the NiFi server.
Could you please assist in investigating the root cause of this discrepancy?

Re: Investigation of Future Timestamps Generated in NiFi Due to Server Timezone Change

MattWho — Thu, 06 Mar 2025 13:51:26 GMT

@NaveenSagar

Welcome to the community.

There is not enough information provided to investigate your issue.

What version of Apache NiFi are you using?
What processor(s) is producing the timestamps in question?
What is the configuration of the processor(s)?

Thank you,

Matt

Re: Investigation of Future Timestamps Generated in NiFi Due to Server Timezone Change

NaveenSagar — Thu, 06 Mar 2025 14:45:38 GMT

Hello @MattWho,
1. We are using NiFi 1.17
2. I used UpdateAttribute Processor
3. Used ${now():format('YYYYMMddHHmmss')} this expression to generate current timestamp.
But for last 3 days of Dec 2024 , we got feature dates as 20251229......etc.

This is a global issue or please assist to find the root cause.
Checked NiFi logs and System logs , all looks good.

Please help.

Re: Investigation of Future Timestamps Generated in NiFi Due to Server Timezone Change

MattWho — Thu, 06 Mar 2025 19:53:03 GMT

@NaveenSagar

Typically when odd timestamp behavior like this is seen it is caused by some custom processor or script being executed within some NiFi dataflow that is timezone in Java or messing with java date which ends impacting the rest of NiFi. So I'd recommend looking at any custom processors or scripts you have being executed to see if any of them is manipulating the Java time. The impacts of doing such things will impact the JVM that all NiFi components are using. Restart would return things to normal until such script or custom processor was executed again.

I can't think of anything else that would impact here.
The now() function returns the current date and time.
So not knowing exactly what was being manipulated and adjusted with regards to system time in the end of December 24, I have no other suggestions. Perhaps whomever was adjusting timezones made a mistake(s) and then corrected them. Are you a NIFi cluster or standalone single NiFi instance? If you are a NiFi cluster was same observation made on every node in the cluster or only impacted FlowFiles on one of the nodes.

Apache NiFi 1.17 is ~3 years old.

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Re: Investigation of Future Timestamps Generated in NiFi Due to Server Timezone Change

NaveenSagar — Fri, 07 Mar 2025 05:21:40 GMT

Hi @MattWho , Thank you so much for reply.
We have a 3 node cluster and it is affected in all 3 nodes.

Re: Investigation of Future Timestamps Generated in NiFi Due to Server Timezone Change

MattWho — Tue, 11 Mar 2025 13:21:00 GMT

@NaveenSagar

I was not able to reproduce in my NiFi (1.26 based). All I can figure is that some changes occurred that impacted the dates in those three days and not afterwards or perhaps the timestamps produced by the ${now():format('YYYYMMddHHmmss')} were manipulated downstream in your dataflow by another processor?

Thanks,
Matt

Re: Investigation of Future Timestamps Generated in NiFi Due to Server Timezone Change

NaveenSagar — Wed, 12 Mar 2025 09:46:16 GMT

Hello @MattWho ,
This occured only for those 3 dates and from next date till today it is working normal.
NOTE: this issue happened in all dev , QA, PROD environments.

Re: Investigation of Future Timestamps Generated in NiFi Due to Server Timezone Change

MattWho — Wed, 12 Mar 2025 18:03:17 GMT

@NaveenSagar

I think I know your issue. You can encounter unexpected result when you do you use the correct simpleDateFormat.

You are using below:

${now():format('YYYYMMddHHmmss')}

It should be:

${now():format('yyyyMMddHHmmss')}

NiFi uses Java Date patterns:
https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html

"y" is used for year

"Y" is used for Week year

Fixing your date format should prevent you from seeing issue in the future.

Thank you,
Matt

Re: Investigation of Future Timestamps Generated in NiFi Due to Server Timezone Change

DianaTorres — Mon, 17 Mar 2025 18:45:38 GMT

@NaveenSagar Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.