Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Filter JSON File content using EvaluateJsonPath

avatar
Explorer

So I'm working with a batch JSON file with the following value :

[
  {
    "eventType": "UPDATE",
    "eventTime": "2021-12-14T12:34:56.789012Z",
    "country": "ES",
    "resourceType": "Party"
  },
  {
    "eventType": "UPDATE",
    "eventTime": "2021-12-14T12:34:56.789012Z",
    "country": "ES",
    "resourceType": "Party"
  },
  {
    "eventType": "UPDATE",
    "eventTime": "2021-12-14T12:34:56.789012Z",
    "country": "FR",
    "resourceType": "Party"
  },
  {
    "eventType": "UPDATE",
    "eventTime": "2021-12-14T12:34:56.789012Z",
    "country": "FR",
    "resourceType": "Party"
  }
]
So I'm working with a batch JSON file with the following value :

[
  {
    "eventType": "UPDATE",
    "eventTime": "2021-12-14T12:34:56.789012Z",
    "country": "ES",
    "resourceType": "Party"
  },
  {
    "eventType": "UPDATE",
    "eventTime": "2021-12-14T12:34:56.789012Z",
    "country": "ES",
    "resourceType": "Party"
  },
  {
    "eventType": "UPDATE",
    "eventTime": "2021-12-14T12:34:56.789012Z",
    "country": "FR",
    "resourceType": "Party"
  },
  {
    "eventType": "UPDATE",
    "eventTime": "2021-12-14T12:34:56.789012Z",
    "country": "FR",
    "resourceType": "Party"
  }
]

I'm able to extract attributes from JSON by using EvaluateJsonPath processor.

UpdateAttribute is the processor where i want to extract the attributes. Please find below snapshot of UpdateAttribute processor when value of "country="ES"

I want get json file like this:

[
  {
    "eventType": "UPDATE",
    "eventTime": "2021-12-14T12:34:56.789012Z",
    "country": "ES",
    "resourceType": "Party"
  },
  {
    "eventType": "UPDATE",
    "eventTime": "2021-12-14T12:34:56.789012Z",
    "country": "ES",
    "resourceType": "Party"
  }
]

1.png2.png3.png

2 REPLIES 2

avatar
Super Guru

Hi,

I dont think the EvaluateJsonPath and RouteOnAttribute will work properly since you have an array of json records in the the incoming flowfiles. You have to split the array using SplitJson processor and then use the EvaluateJsonPath->RouteOnAttributes to get the expected result. However there is better and more efficient option than using the mentioned processors (Split->Evaluate->Route), you can use just QueryRecords to filter the array and isolate the needed records. You can see example on how to use QueryRecords here: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.19.1/org.apach...

 

If that helps , please accept solution.

 

avatar
Contributor

As @SAMSAL pointed out, queryrecord is the best and easiest method to use.
Here, i have created a sample flow for your use as pix attached.
I hope it helps.
cld-1.png