Support Questions
Find answers, ask questions, and share your expertise

Nifi TImeStamp check for JSON record?

Nifi TImeStamp check for JSON record?

Explorer

Hi All,

My flow is moving a lot of JSON files in the following format:

[{
        "name": "Don",
        "id": 15 
        "opened": "2019-03-26T05:47:14.670Z",
        "a": "2019-03-26T06:31:45.133Z",
        "closed": "2019-03-26T06:33:56.960Z",
        "periodA": "00:44:31",
        "periodN": "00:02:11"
    }, {
        "name": "Don,
        "opened": "2019-03-26T05:57:43.763Z",
        "a": "2019-03-26T06:13:48.150Z",
        "closed": "2019-03-26T06:31:16.273Z",
        "periodA": "00:16:05",
        "periodN": "00:17:28"
    }]


Is there a way to check if timestamp fields are in ISO8601 format, and if not convert it accordingly? What is the correct way of doing this? The utlimate result will be pushed to Kafka in AVRO format.

3 REPLIES 3

Re: Nifi TImeStamp check for JSON record?

Explorer

@Matt Burgess do you mind taking a look? Is this something that can be done with Validate Record?

Re: Nifi TImeStamp check for JSON record?

Super Guru

@Rosa Negra

Yes, you are in right track we need to use Validate Record processor to perform this check and while defining avro schema you need to use Avro logical types for timestamp type and we need to keep the format of the timestamp field also.

Refer to this template for defining avro schema with logical types.

Re: Nifi TImeStamp check for JSON record?

New Contributor

Iam also having the same issue.I have multiple fields and multiple files coming with this format. Any generic way to handle this?. Are there any sample for the validate processor and conversion of these fields?. Thanks in advance.