Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NiFi : combine flow file attribute and content

avatar
Rising Star

Hello,

I am puzzelling for some time if there is possibility to refer the both attributes and content of flow file and transform the content by combining both.
Example:
Assume a flow file with uuid=2c8a7845-8567-4eed-97ee-4dcd1c18668e and the json content

{
   "data":"some business data"
}

 

The result of the transformation should be a content like

{
   "nifi-filename": "2c8a7845-8567-4eed-97ee-4dcd1c18668e",
   "nifi-content": {
       "data": "some business data"
   }
}

 

How can I reach that.?

 

Best regards
Jaro

1 ACCEPTED SOLUTION

avatar
Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
4 REPLIES 4

avatar
Super Guru

@Jarinek   Yes, this is totally possible in NiFi.   Traditionally one method to do this could be something like EvaluateJsonPath to get some payload json values to attributes and then use AttributesToJson to reform a new json object of all the attributes you have created.   In newer versions of Nifi you have some other options with QueryRecord, UpdateRecord and the Json Record Readers, Record Writers.  I think another option is also custom ExecuteScript and JoltTransform on JSON.

 

 

My suggestion would be to research NiFi + JSON and begin with some simple examples.  Once you have some experience with basic examples, begin to transform them to suit your use case.   I also suggest that you try it in different ways before deciding on one.  For example,  you may build a flow and get it working (evaluateJsonPath), but improve it over time based on new nifi versions and its capabilities (Use JSON Record Reader/Writers).

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven @ DFHZ

avatar
Rising Star

Well, EvaluateJsonPath sounds promising but it looks like that EvaluateJsonPath is somehow limitated.
It is unable to extract json objects, only single attributes, which makes it impossible to use for slightly more complex data.
Let's suppose this.

Input:
{
   whatever known or unknown json nested structures here
}

Desired output
{
    "nifi-filename": "2c8a7845-8567-4eed-97ee-4dcd1c18668e",
    "nifi-content": {
       whatever known or unknown json possibly structures or sub-structures of the input message here
    }
}

avatar
Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Rising Star

Actually, both replies can be considered as valid. I confirmed that one, which better fits to my use case.