question Re: NiFi updateRecord - dealing with Blank fields in Support Questions

NiFi updateRecord - dealing with Blank fields

Griggsy — Fri, 28 Oct 2022 09:39:26 GMT

Hello all

I have a slightly annoying problem in NiFi, I have some JSON records that have a field 'date' nested under 'sources' that I want to turn to Epoch time. Using updateRecord I am able to do this using the following config:

As you can see where this field is present it works, however sometimes this field is not present, in these cases the record value is blank:

To do deal with these blank field I did some searching online and found the following 'isBlank' could be used in updateRecord property to deal with blank fields:

The outcome of this fixes the blank field issue... but, doesn't convert the present field values to epoch, see below:

Does anyone know how I can get these two actions to perform at the same time?

Thanks in advance.

Andy

Re: NiFi updateRecord - dealing with Blank fields

steven-matison — Fri, 28 Oct 2022 12:31:33 GMT

@Griggsy I would have to have working sample to test and provide better solution, that said i think you need to use function inside of the value, not the property. There you could also use ifElse making the NiFi Expression language operate for both scenarios. Chaining expressions together can be a challenge but should get you where you need to go.

Re: NiFi updateRecord - dealing with Blank fields

Griggsy — Mon, 31 Oct 2022 09:16:51 GMT

Hi @steven-matison

Here is a flow file example, as you can see the date field is only present in the array some of the time, what I want to achieve is, if the date is present I need it in epoch. If the date is not present for the field to be 'null,' (not "null" as a sting/in brackets) or ignored altogether.

{
"example": "cloudera",
"risk_rating": "low",
"observed" false,
"sources": [ {
"date": "2020-06-12T12:00:00.000Z",
"unique_id": "some_value",
"source_name": "cloudera",
"url": "https://www.cloudera.com",
"description": "example_log",
},{
"unique_id": "some_other_value",
"source_name": "google",
"url": "https://www.google.com",
"description": "example_log_without_date"
}],
"report_confidence": "confirmed",
"base_score" 7.8,
"authentication": "none"
}

Re: NiFi updateRecord - dealing with Blank fields

Griggsy — Mon, 31 Oct 2022 16:33:15 GMT

@steven-matison

I have been trying to get ifElse working for me but the below gives me an empty string > ""

And this gives me null as a string > "null"

is there a way to return null not as a string?