NiFi UpdateRecord processor is not updating JSON path variable.

[#1]

I'm working on a NIFI flow to update incoming JSON file data using UpdateRecord processor.

Incoming JSON sample data is provided below:

[ {
  "created_at" : "Tue May 08 16:05:42 CDT 2018",
  "id_store" : 1,
  "event_type" : "generated",
  "id_transaction" : "7276464206868934013",
  "id_product" : 1,
  "value_product" : 1100152002,
  "prod_desc" : "MapRecord[{id_product=appleA}]"
} ]

I used HBaseLookup Service to lookup "id_product" value from incoming data in HBase database to derive "prod_desc" value. HBase database lookup is working perfectly. The lookup process returned the value "MapRecord[{id_product=appleA}]" under "prod_desc" field for id_product value of "1". Now I'm using UpdateRecord processor to remove everything else except the value "appleA". Attached is the screenshot for UpdateRecord processor setting. It seems not replacing the value as I expected, instead the value "null" is returned under "prod_desc" field. I'm not sure what went wrong in my setting.

[#2 · Accepted solution]

@Winnie Philip

Make sure in Update Record processor Record Reader controller service having prod_desc column name is defined in the avro schema/registry.
if the Record Reader is not having prod_desc column name is defined then Reader not able to read the incoming json message for prod_desc column, which will result writer with null value for prod_desc column value.
Instead of using two update record processors you can use the below dynamic property value

substringBefore(substringAfter( /prod_desc, '=' ),'}')

Update record processor configs:

Sample Record Reader for update record processor:

Avro Schema with prod_desc column in it

{
"namespace": "nifi",
"name": "person",
"type": "record",
"fields": [
{ "name": "created_at", "type": ["null","string"] },
{ "name": "id_store", "type": ["null","int"] },
-----
-----
{ "name": "prod_desc", "type": ["null","string" ]}]}

-

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

[#3]

Thank you, Shu. This worked perfectly, thanks for the additional info as well.