Dynamic property naming in NiFi

[#1]

Hi there,

I'm in a situation where I need to be able to set the name of a property using Expression Language. Is there any way to do that?

I try to create an attribute using an UpdateAttribute processor and put in ${property1} as the property name. Now, property1 is in reference to an existing attribute containing the value "property2". What I'd hope is that the flowfile has a new property added named "property2" but what I end up with is a new property literally named "${property1}".

Is this even possible?

Thanks,

JP

[#2]

Maybe there's another way to ask this; I shouldn't pre-suppose dynamic naming is necessary.

Say I have this JSON document:

{ "boxes": [{ "box": { "oranges": 2, "apples": 5 } }, { "box": { "oranges": 2, "bananas": 7 } } ] }

I need to then do some work to create something like this:

{ "boxes": [{ "box": 1, "fruits": 2, "count": 7 }, { "box": 2, "fruits": 2, "count": 9 } ] }

My thinking was first split by box to get a "box" attribute and 2 flowfiles. Then split by contents of each box to create 4 flowfiles each with a "fruit" attribute. But how do I merge back to construct pieces of the final output? The attribute names would be the same with different values. I need for the attribute names to be different (like box.1, box.2, fruit.1, fruit.2) which would be easy using fragment indexes, but I can't seem to name properties dynamically like that. Is there an alternative?

Does that make any sense?

Thanks,

JP

[#3]

Alternately, is there a way to get the size of a JSON array using EvaluateJSONPath?

[#4 · Accepted solution]

Hi,

Take a look at the JoltTransformJSON processor, it is capable to do the exact transformation you need.

More info about Jolt:

https://github.com/bazaarvoice/jolt

http://jolt-demo.appspot.com/

Cheers.

[#5]

@JP

One way of doing it is using ExecuteScript Processor. Follows a sample solution.

I generated a flow file using GenerateFlowFile Processor which has an attribute named "Property1" with the value set to "Property2".

Now, this goes to an ExecuteScript Processor where I am using a Groovy script to create a new property "on the fly"!

Follows the groovy script which is creating the property on the fly.

flowFile = session.get()
if(!flowFile) return
myAttr = flowFile.getAttribute('property1')
flowFile = session.putAttribute(flowFile, myAttr, 'myValue')
session.transfer(flowFile, REL_SUCCESS)

The flow file coming out of this processor looks something like this.

Hope that helps!

[#6]

As per Taras' answer; a JOLT transform ended up being exactly what we needed.