NiFi: insert timestamp to json

[#1]

json.png

I want to insert timestamp in json as in pictures:

For example:

{"submit_date": "2017-09-11 12:28:57.0", "request_type": "change", "origin": "monitoring", "description": "Start Event Process from Zabbix"}

expected:

{"submit_date": "2017-09-11 12:28:57.0", "request_type": "change", "origin": "monitoring", "description": "Start Event Process from Zabbix", "timestamp":"2018-08-30 08:16:18.970+0000"}

[#2]

@Thuy Le

There are 2 ways to do this:

1. If you are using submit_date to get the timestamp
2. If you are using now() for the timestamp

I assume you want to use 2, but have included both just in case. In both cases you will need to append the "+0000" or other offset if it is known.

The expression language is as follows:

1. ${submit_date:toDate("yyyy-MM-dd-HH:mm:ss.S"):format("yyyy-MM-dd-HH:mm:ss.SSS")}+0000
2. ${now():format("yyyy-MM-dd-HH:mm:ss.SSS")}+0000

You can find the necessary documentation here:

https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#dates

If this answer is helpful please choose ACCEPT to mark the question resolved.

[#3]

Hi Steven Matison,

With processors will I need to insert date to JSON?

[#4]

@Thuy Le

Use UpdateRecord processor and add the new property as

/timestamp

value as

${now():format("yyyy-MM-dd HH:mm:ss.SSS")}+0000

Replacement value strategy as

Literal value

Refer to this link for configuring/usage of Update Record processor.

In UpdateRecord processor configure RecordReader as JsonTreeReader and Record Writer as JsonSetWriter and include your new timestamp field with value as string in Avro Schema.Then processor will adds the timestamp field into output flowfile.

By using UpdateRecord processor you can either use Record Path value to generate timestamp field value (or) literal value to get the timestamp field value.

-

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

[#5]

Hi Shu,

Thank you so much for your suggestions, Do you have any other solutions without use Update Record processor?

I used it then when I insert json to Elasticsearch it can't convert string (submit_date, timestamp) to datetime.

The main case here we have json (from rabbitmq) as:

{"submit_date": "2017-09-11 12:28:57.0", "request_type": "change", "origin": "monitoring", "description": "Start Event Process from Zabbix"}

then we insert timestamp to json :

{"submit_date": "2017-09-11 12:28:57.0", "request_type": "change", "origin": "monitoring", "description": "Start Event Process from Zabbix", "timestamp":"2018-08-30 08:16:18.970+0000"}

then insert data to Elasticsearch (they have to understand submit_date, timestamp as datetime)