NiFi - RecordProcessing - question about syntax of RecordPath

[#1]

Hi all, I'm still working on a solution for this problem

https://community.hortonworks.com/questions/248972/nifi-script-for-json-content-manipulation-before-...

Following the recommendation of @Rosa Negra I try to do this with record-processing along this article of @Shu (thanks for this)

https://community.hortonworks.com/content/kbentry/189642/update-the-contents-of-flowfile-by-using-up...

Input-JSON

{
  "user": "Justen",
  "stand": "20190702121621",
  "table": [
    {
      "zn": 1,
      "elem_user": "Meier",
      "elem_stand": "20190705081410",
      "dnr": "(0)",
      "group": "1"
    },
    {
      "elem_stand": "20190706201918",
      "zn": 2,
      "elem_user": "Schmidt",
      "dnr": "(0)",
      "group": "2"
    }
  ]
}

Desired Output-JSON

{
        "user": "Justen",
        "stand": "2019-07-02 10:16:21",
        "table": [
            {
                "zn": 1,
                "elem_user": "Meier",
                "elem_stand": "2019-07-05 06:14:10",
                "dnr": "(0)",
                "group": "1",
                "dnr_group": "V-00-01"
            },
            {
                "elem_stand": "2019-07-06 18:19:18",
                "zn": 2,
                "elem_user": "Schmidt",
                "dnr": "(0)",
                "group": "2",
                "dnr_group": "V-00-02"
            }
        ]
    }

actual Output-JSON of my flow

[ {
  "name" : "Justen",
  "stand" : "2019-07-02 10:16:21.000",
  "table" : [ {
    "zn" : 1,
    "elem_user" : "Meier",
    "elem_stand" : "2019-07-05 06:14:10.000",
    "dnr" : "(0)",
    "group" : "1",
    "dnr_group" : "(0)(0)12"                   ========> first expected "(0) 1"; ideal case "V-00-01"
  }, {
    "zn" : 2,
    "elem_user" : "Schmidt",
    "elem_stand" : "2019-07-06 18:19:18.000",
    "dnr" : "(0)",
    "group" : "2",
    "dnr_group" : "(0)(0)12"                   ========> first expected "(0) 2"; ideal case "V-00-02"
  } ]
} ]

I had success with formatting the timestamps (stand, elem_stand)!

But I can't find the rigt syntax for RecordPath setting "dnr_group".

All examples here https://nifi.apache.org/docs.html in Apache NiFi RecordPath Guide don't show how to use functions working with elements of an array.

In the second UpdateRecord (Change Value - Record Path) I tried "Property" /table[0..-1]/dnr_group for access on all elements which works, but at "Value" I don't know how to reference the elements dnr and group of the single entry.

My aim is still to build this expression which works in EL...

${DNR:substringAfter('('):substringBefore(')'):toNumber()
                            :lt( 10 )
                                     :ifElse(${DNR:substringAfter('('):substringBefore(')'):prepend('0')},
                                             ${DNR:substringAfter('('):substringBefore(')')})
                            :append('-'):append(${GROUP:toNumber()
                            :lt( 10 )
                                     :ifElse(${GROUP:prepend('0')},
                                             ${GROUP})})
    :prepend('V-')}

Please have a look on the attachment which shows anything I defined in NiFi.

Thanks for any help!

[#2]

@ Justen try using evalate json path... https://jsonpath.com/

[#3]

@Rosa Negra

Tried with Jsonpath-expression, but UpdateRecord doesn't like it even though the expression is correct. I suppose it has to be RecordPath-expression here.

UpdateRecord[id=5eb434f8-e995-123d-857f-704c2cff2c0c] Failed to process StandardFlowFileRecord[uuid=a7827a3b-566d-4470-832f-14c62dde7fc3,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1562840736703-7, container=default, section=7], offset=405512, length=283],offset=0,name=IJ_TEST,size=283]; 
will route to failure: Unexpected token ',' at line 1, column 22. Query: concat($..table..dnr, $..table..group)

Even if the Jsonpath-expression would have been accepted the problem remains still the same. Online validator result is also [ "(0)", "(0)" ] I suppose I have to point on the single entry and not to all entries of the array. Sorry can't explain it better...

[#4]

@ Justen as far as the schemas go, try using SimpleKeyLookupService controller service, create two new properties that will point to AvroSchemaRegistry controller service. That way you will call for ${schema.name} attribute in your processors instead of hardcoding the schemas into the processor directly.

[#5]

@Rosa Negra
Thanks for this info. I will go into this as soon as I know that record-processing is suitable approach for this.

[#6]

@ Justen use EvaluateJsonPath to extract the record into an attribute, apply the changes and push it back in with update record processor

[#7]

@Rosa Negra

OK, processor EvaluateJsonPath extracted the content to attribute FF_CONTENT.

To do the string manipulation within the attribute I suppose I have to use jsonPath function. But now I'm facing the same problem as before with RecordPath-syntax. I can't find the right syntax now to make the string manipulation within the attribute.

I feel like going around in circles...