Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to filter an array in NiFi RecordPath using a value from another field?

How to filter an array in NiFi RecordPath using a value from another field?

New Contributor

Hi,

Is there a way to specify an array index as an expression using another field value within the record.

Let me clarify:

I have following performance data XML (simplified)

<measCollecFile>
  <measData>
    <measInfo>
      <measType p="1">kpi1</measType>
      <measType p="2">kpi2</measType>
      <measType p="3">kpi1</measType>
      <measValue measObjLdn="ManagedElement1">
        <r p="1">100</r>
      </measValue>
    </measInfo>
  </measData>
</measCollecFile>
  1. <measInfo> and <r> are arrays.
  2. <r> is value for kpi that referred by <measType>, correlated by attribute 'p'.

I have managed to convert the above XML to avro as follow:

{
  "measData": {
    "measInfo": [
      {
        "measType": [
          {
            "p": {
              "string": "1"
            },
            "text": {
              "string": "kpi1"
            }
          },
          {
            "p": {
              "string": "2"
            },
            "text": {
              "string": "kpi2"
            }
          },
          {
            "p": {
              "string": "3"
            },
            "text": {
              "string": "kpi3"
            }
          }
        ],
        "measValue": {
          "measObjLdn": {
            "string": "ManagedElement1"
          },
          "r": [
            {
              "text": {
                "string": "100"
              },
              "p": {
                "string": "1"
              }
            }
          ]
        }
      }
    ]
  }
}

After parsing, I ran it thru ForkRecord processor to split each <r> into its own record. This way I can be sure there will no more than one instance of <r> for each record.

Now, I want to flatten this nested structure to flat records.

Using UpdateRecord:

  • /measObjLdn : /measData/measInfo[0]/measValue/measObjLdn
  • /measType : /measData/measInfo[0]/measType[0]/text
  • /measValue : /measData/measInfo[0]/measValue/r[0]/text

All these works. However /measType is clearly not I want (above will always pick the first in array). What I want is:

/measData/measInfo[0]/measType[X]/text, X should be from /measData/measInfo[0]/measValue/r[0]/p

If I directly write it as /measData/measInfo[0]/measType[ /measData/measInfo[0]/measValue/r[0]/p]/text, it complains unexpected '[' and so on.

How can this be expressed? Thanks.

2 REPLIES 2

Re: How to filter an array in NiFi RecordPath using a value from another field?

New Contributor

thanks for posting.

Highlighted

Re: How to filter an array in NiFi RecordPath using a value from another field?

New Contributor

I believe you wanted to add it as a comment and not as an answer.

Don't have an account?
Coming from Hortonworks? Activate your account here