Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi processor: Convert string(datetime format) to long(unix_timestamp) in CSV field

Solved Go to solution
Highlighted

NiFi processor: Convert string(datetime format) to long(unix_timestamp) in CSV field

Contributor

Hello,

Is there a processor help with converting a string(Datetime format: 2017-01-18 13:28:17) to long(133434444545) in a csv field?

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NiFi processor: Convert string(datetime format) to long(unix_timestamp) in CSV field

Guru

Hi @Alvin Jin

I haven't seen a special processor to do this yet, but, I've seen a few examples where this can be done using the NiFi Expression Language. These two links should get you going.

https://community.hortonworks.com/questions/55610/converting-datetime-to-unixepoch-time-milliseconds...

https://community.hortonworks.com/questions/48905/date-problems-with-convertjsontosql-or-putsql-in-n...

View solution in original post

5 REPLIES 5
Highlighted

Re: NiFi processor: Convert string(datetime format) to long(unix_timestamp) in CSV field

Guru

Hi @Alvin Jin

I haven't seen a special processor to do this yet, but, I've seen a few examples where this can be done using the NiFi Expression Language. These two links should get you going.

https://community.hortonworks.com/questions/55610/converting-datetime-to-unixepoch-time-milliseconds...

https://community.hortonworks.com/questions/48905/date-problems-with-convertjsontosql-or-putsql-in-n...

View solution in original post

Highlighted

Re: NiFi processor: Convert string(datetime format) to long(unix_timestamp) in CSV field

Contributor

@Sonu Sahi

Thank you for your response.

${now():toNumber():format('yyyy-MM-dd')}

It seems above expression works for attributes.

How to use it to convert a column in csv file?

Thanks.

Highlighted

Re: NiFi processor: Convert string(datetime format) to long(unix_timestamp) in CSV field

Guru

Hi @Alvin Jin

You could use a combination of ExtractText and ReplaceText with some RegEx, though it might be a bit tedious. There is a good example here with screens that you can use as a guide:

https://community.hortonworks.com/articles/66861/nifi-etl-removing-columns-filtering-rows-changing.h...

Highlighted

Re: NiFi processor: Convert string(datetime format) to long(unix_timestamp) in CSV field

Super Guru

As of NiFi 1.3.0, you can use UpdateRecord for this. If your incoming field name is "createdOn", you can add a user-defined property named "/createdOn" whose value is the following:

${field.value:toDate('yyyy-mm-dd HH:mm:ss.SSS'):toNumber()}

Note that you may need to change the type of createdOn from String (in the Reader's schema) to Long (in the Writer's schema).

Re: NiFi processor: Convert string(datetime format) to long(unix_timestamp) in CSV field

How would I use this with a literal letter in my timestamp format? My timestamp format looks like this:

yyyy-mm-dd'T'HH:mm:ss.SSS
Don't have an account?
Coming from Hortonworks? Activate your account here