Support Questions

Find answers, ask questions, and share your expertise

NiFi updateRecord - dealing with Blank fields

avatar
Contributor

Hello all

I have a slightly annoying problem in NiFi, I have some JSON records that have a field 'date' nested under 'sources' that I want to turn to Epoch time. Using updateRecord I am able to do this using the following config:

Griggsy_0-1666948710849.png

As you can see where this field is present it works, however sometimes this field is not present, in these cases the record value is blank:

Griggsy_1-1666949308722.png

To do deal with these blank field I did some searching online and found the following 'isBlank' could be used in updateRecord property to deal with blank fields:

Griggsy_2-1666949530183.png

The outcome of this fixes the blank field issue... but, doesn't convert the present field values to epoch, see below:

Griggsy_3-1666949742086.png

Does anyone know how I can get these two actions to perform at the same time? 

Thanks in advance.

 

Andy

 

3 REPLIES 3

avatar

@Griggsy  I would have to have working sample to test and provide better solution, that said i think you need to use function inside of the value, not the property.   There you could also use ifElse making the NiFi Expression language operate for both scenarios.   Chaining expressions together can be a challenge but should get you where you need to go.

avatar
Contributor

Hi @steven-matison 

 

Here is a flow file example, as you can see the date field is only present in the array some of the time, what I want to achieve is, if the date is present I need it in epoch. If the date is not present for the field to be 'null,' (not "null" as a sting/in brackets) or ignored altogether.

 

{
"example": "cloudera",
"risk_rating": "low",
"observed" false,
"sources": [ {
"date": "2020-06-12T12:00:00.000Z",
"unique_id": "some_value",
"source_name": "cloudera",
"url": "https://www.cloudera.com",
"description": "example_log",
},{
"unique_id": "some_other_value",
"source_name": "google",
"url": "https://www.google.com",
"description": "example_log_without_date"
}],
"report_confidence": "confirmed",
"base_score" 7.8,
"authentication": "none"
}

avatar
Contributor

@steven-matison 

I have been trying to get ifElse working for me but the below gives me an empty string > ""

Griggsy_0-1667233807380.png

And this gives me null as a string > "null"

Griggsy_1-1667233936356.png

is there a way to return null not as a string?