Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

NiFi updateRecord - dealing with Blank fields

Contributor

Hello all

I have a slightly annoying problem in NiFi, I have some JSON records that have a field 'date' nested under 'sources' that I want to turn to Epoch time. Using updateRecord I am able to do this using the following config:

Griggsy_0-1666948710849.png

As you can see where this field is present it works, however sometimes this field is not present, in these cases the record value is blank:

Griggsy_1-1666949308722.png

To do deal with these blank field I did some searching online and found the following 'isBlank' could be used in updateRecord property to deal with blank fields:

Griggsy_2-1666949530183.png

The outcome of this fixes the blank field issue... but, doesn't convert the present field values to epoch, see below:

Griggsy_3-1666949742086.png

Does anyone know how I can get these two actions to perform at the same time? 

Thanks in advance.

 

Andy

 

3 REPLIES 3

Super Collaborator

@Griggsy  I would have to have working sample to test and provide better solution, that said i think you need to use function inside of the value, not the property.   There you could also use ifElse making the NiFi Expression language operate for both scenarios.   Chaining expressions together can be a challenge but should get you where you need to go.

Contributor

Hi @steven-matison 

 

Here is a flow file example, as you can see the date field is only present in the array some of the time, what I want to achieve is, if the date is present I need it in epoch. If the date is not present for the field to be 'null,' (not "null" as a sting/in brackets) or ignored altogether.

 

{
"example": "cloudera",
"risk_rating": "low",
"observed" false,
"sources": [ {
"date": "2020-06-12T12:00:00.000Z",
"unique_id": "some_value",
"source_name": "cloudera",
"url": "https://www.cloudera.com",
"description": "example_log",
},{
"unique_id": "some_other_value",
"source_name": "google",
"url": "https://www.google.com",
"description": "example_log_without_date"
}],
"report_confidence": "confirmed",
"base_score" 7.8,
"authentication": "none"
}

Contributor

@steven-matison 

I have been trying to get ifElse working for me but the below gives me an empty string > ""

Griggsy_0-1667233807380.png

And this gives me null as a string > "null"

Griggsy_1-1667233936356.png

is there a way to return null not as a string? 

 

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.