Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to create a new attribute with value as the content of the flowfile?

How to create a new attribute with value as the content of the flowfile?

New Contributor

found this article:

https://community.hortonworks.com/questions/52055/how-to-create-a-new-attribute-with-value-as-the-co...

but it doesn't seem to work

I added new property "message.body" to ReplaceText with value of "(.*)" and i see an error: "message.body is invalid when validated against (.*) because message.body is not supported property"

4 REPLIES 4

Re: How to create a new attribute with value as the content of the flowfile?

Did you mean ExtractText instead of ReplaceText?

Re: How to create a new attribute with value as the content of the flowfile?

Master Guru

@Victor L

In addition to what @Matt Burgess mention above about using ExtractText processor instead of ReplaceText processor to pull FlowFile content out into a FlowFile attribute, I want to caution you about extracting large amounts of content to attributes on a FlowFile.

FlowFile attributes are held in the NiFi JVM heap memory. Extracting large amounts of content into attributes can lead to Out of Memory (OOM) conditions with your JVM. I noticed you are looking to use .* to extract the entire content to an attribute.


Thank you,

Matt

Re: How to create a new attribute with value as the content of the flowfile?

New Contributor

Seems as value comes garbled: if my content is:

[{“username”: “one”, “create_date":1550500579036, “role”: “tech lead”}, {“username”: “two”, “create_date":1550500579100, “role”: “user”}]

than i have 3 attributes:
body
:1550500579036, “role”: “tech lead”}, {“username”: “two”, “create_date


body.0
":1550500579036, “role”: “tech lead”}, {“username”: “two”, “create_date"


body.1
:1550500579036, “role”: “tech lead”}, {“username”: “two”, “create_date
Highlighted

Re: How to create a new attribute with value as the content of the flowfile?

Master Guru

@Victor L

Please share your ExtractText processor configuration and the java regular expression you are using.

I just tested using your json example above and was able to extract complete content without issue.

I used ^\[(.*)\]$ and also used ^\[(.*?}),(.*?)\]$. Both worked with latter creating a different attribute for each user json.


Thanks,

Matt