Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Dynamic property naming in NiFi

Explorer

Hi there,

I'm in a situation where I need to be able to set the name of a property using Expression Language. Is there any way to do that?

I try to create an attribute using an UpdateAttribute processor and put in ${property1} as the property name. Now, property1 is in reference to an existing attribute containing the value "property2". What I'd hope is that the flowfile has a new property added named "property2" but what I end up with is a new property literally named "${property1}".

Is this even possible?

Thanks,

JP

1 ACCEPTED SOLUTION

New Contributor

Hi,

Take a look at the JoltTransformJSON processor, it is capable to do the exact transformation you need.

More info about Jolt:

https://github.com/bazaarvoice/jolt

http://jolt-demo.appspot.com/

Cheers.

View solution in original post

5 REPLIES 5

Explorer

Maybe there's another way to ask this; I shouldn't pre-suppose dynamic naming is necessary.

Say I have this JSON document:

{ "boxes": [{ "box": { "oranges": 2, "apples": 5 } }, { "box": { "oranges": 2, "bananas": 7 } } ] }

I need to then do some work to create something like this:

{ "boxes": [{ "box": 1, "fruits": 2, "count": 7 }, { "box": 2, "fruits": 2, "count": 9 } ] }

My thinking was first split by box to get a "box" attribute and 2 flowfiles. Then split by contents of each box to create 4 flowfiles each with a "fruit" attribute. But how do I merge back to construct pieces of the final output? The attribute names would be the same with different values. I need for the attribute names to be different (like box.1, box.2, fruit.1, fruit.2) which would be easy using fragment indexes, but I can't seem to name properties dynamically like that. Is there an alternative?

Does that make any sense?

Thanks,

JP

Explorer

Alternately, is there a way to get the size of a JSON array using EvaluateJSONPath?

New Contributor

Hi,

Take a look at the JoltTransformJSON processor, it is capable to do the exact transformation you need.

More info about Jolt:

https://github.com/bazaarvoice/jolt

http://jolt-demo.appspot.com/

Cheers.

@JP

One way of doing it is using ExecuteScript Processor. Follows a sample solution.

I generated a flow file using GenerateFlowFile Processor which has an attribute named "Property1" with the value set to "Property2".

64755-screen-shot-2018-03-22-at-121100-am.png

Now, this goes to an ExecuteScript Processor where I am using a Groovy script to create a new property "on the fly"!

64756-screen-shot-2018-03-22-at-121342-am.png

Follows the groovy script which is creating the property on the fly.

flowFile = session.get()
if(!flowFile) return
myAttr = flowFile.getAttribute('property1')
flowFile = session.putAttribute(flowFile, myAttr, 'myValue')
session.transfer(flowFile, REL_SUCCESS)

The flow file coming out of this processor looks something like this.

64757-screen-shot-2018-03-22-at-121458-am.png

Hope that helps!

Explorer

As per Taras' answer; a JOLT transform ended up being exactly what we needed.