question Re: Dynamic property naming in NiFi in Support Questions

Dynamic property naming in NiFi

james_poriz — Sat, 10 Feb 2018 04:04:30 GMT

Hi there,

I'm in a situation where I need to be able to set the name of a property using Expression Language. Is there any way to do that?

I try to create an attribute using an UpdateAttribute processor and put in ${property1} as the property name. Now, property1 is in reference to an existing attribute containing the value "property2". What I'd hope is that the flowfile has a new property added named "property2" but what I end up with is a new property literally named "${property1}".

Is this even possible?

Thanks,

Re: Dynamic property naming in NiFi

james_poriz — Sat, 10 Feb 2018 04:42:06 GMT

Maybe there's another way to ask this; I shouldn't pre-suppose dynamic naming is necessary.

Say I have this JSON document:

{ "boxes": [{ "box": { "oranges": 2, "apples": 5 } }, { "box": { "oranges": 2, "bananas": 7 } } ] }

I need to then do some work to create something like this:

{ "boxes": [{ "box": 1, "fruits": 2, "count": 7 }, { "box": 2, "fruits": 2, "count": 9 } ] }

My thinking was first split by box to get a "box" attribute and 2 flowfiles. Then split by contents of each box to create 4 flowfiles each with a "fruit" attribute. But how do I merge back to construct pieces of the final output? The attribute names would be the same with different values. I need for the attribute names to be different (like box.1, box.2, fruit.1, fruit.2) which would be easy using fragment indexes, but I can't seem to name properties dynamically like that. Is there an alternative?

Does that make any sense?

Thanks,

Re: Dynamic property naming in NiFi

james_poriz — Sat, 10 Feb 2018 05:35:21 GMT

Alternately, is there a way to get the size of a JSON array using EvaluateJSONPath?

Re: Dynamic property naming in NiFi

quakerlive — Thu, 22 Mar 2018 00:44:49 GMT

Hi,

Take a look at the JoltTransformJSON processor, it is capable to do the exact transformation you need.

More info about Jolt:

https://github.com/bazaarvoice/jolt

http://jolt-demo.appspot.com/

Cheers.

Re: Dynamic property naming in NiFi

RahulSoni — Sun, 18 Aug 2019 03:07:40 GMT

@JP

One way of doing it is using ExecuteScript Processor. Follows a sample solution.

I generated a flow file using GenerateFlowFile Processor which has an attribute named "Property1" with the value set to "Property2".

Now, this goes to an ExecuteScript Processor where I am using a Groovy script to create a new property "on the fly"!

Follows the groovy script which is creating the property on the fly.

flowFile = session.get()
if(!flowFile) return
myAttr = flowFile.getAttribute('property1')
flowFile = session.putAttribute(flowFile, myAttr, 'myValue')
session.transfer(flowFile, REL_SUCCESS)

The flow file coming out of this processor looks something like this.

Hope that helps!

Re: Dynamic property naming in NiFi

james_poriz — Thu, 22 Mar 2018 20:22:36 GMT

As per Taras' answer; a JOLT transform ended up being exactly what we needed.