Skip to main content
Jarinek
Jarinek
Gold ⭐️
August 27, 2020
Solved

NiFi : combine flow file attribute and content

Hello,

I am puzzelling for some time if there is possibility to refer the both attributes and content of flow file and transform the content by combining both.
Example:
Assume a flow file with uuid=2c8a7845-8567-4eed-97ee-4dcd1c18668e and the json content

{
   "data":"some business data"
}

 

The result of the transformation should be a content like

{
   "nifi-filename": "2c8a7845-8567-4eed-97ee-4dcd1c18668e",
   "nifi-content": {
       "data": "some business data"
   }
}

 

How can I reach that.?

 

Best regards
Jaro

Kezia
KeziaAnswer
Gold ⭐️
August 31, 2020

Hello,

 

You can create a nested node "nifi-content" using the ReplaceText processor.

You just need to configure the Replacement Value field :

 

{"nifi-content":$1}

 

 

Then just add a new node "filename" using the JoltTransformJSON processor with following Jolt specification :

 

[
  {
    "operation": "default",
    "spec": {
      "filename": "${filename}"
    }
}, {
    "operation": "default",
    "spec": {
      "*": "&"
    }
}
]

 

 

Sans titre.png

 

Maybe you can create the "nifi-content" node and add the "filename" node only using one JoltTransform JSON ; but i'm not very experienced using Jolt and I did not find out how.

View in thread

4 replies

stevenmatison
stevenmatison
Platinum ⭐️⭐️
August 27, 2020

@Jarinek   Yes, this is totally possible in NiFi.   Traditionally one method to do this could be something like EvaluateJsonPath to get some payload json values to attributes and then use AttributesToJson to reform a new json object of all the attributes you have created.   In newer versions of Nifi you have some other options with QueryRecord, UpdateRecord and the Json Record Readers, Record Writers.  I think another option is also custom ExecuteScript and JoltTransform on JSON.

 

 

My suggestion would be to research NiFi + JSON and begin with some simple examples.  Once you have some experience with basic examples, begin to transform them to suit your use case.   I also suggest that you try it in different ways before deciding on one.  For example,  you may build a flow and get it working (evaluateJsonPath), but improve it over time based on new nifi versions and its capabilities (Use JSON Record Reader/Writers).

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven @ DFHZ

Translate
Jarinek
JarinekAuthor
Gold ⭐️
August 28, 2020

Well, EvaluateJsonPath sounds promising but it looks like that EvaluateJsonPath is somehow limitated.
It is unable to extract json objects, only single attributes, which makes it impossible to use for slightly more complex data.
Let's suppose this.

Input:
{
   whatever known or unknown json nested structures here
}

Desired output
{
    "nifi-filename": "2c8a7845-8567-4eed-97ee-4dcd1c18668e",
    "nifi-content": {
       whatever known or unknown json possibly structures or sub-structures of the input message here
    }
}

Translate
Kezia
KeziaAnswer
Gold ⭐️
August 31, 2020

Hello,

 

You can create a nested node "nifi-content" using the ReplaceText processor.

You just need to configure the Replacement Value field :

 

{"nifi-content":$1}

 

 

Then just add a new node "filename" using the JoltTransformJSON processor with following Jolt specification :

 

[
  {
    "operation": "default",
    "spec": {
      "filename": "${filename}"
    }
}, {
    "operation": "default",
    "spec": {
      "*": "&"
    }
}
]

 

 

Sans titre.png

 

Maybe you can create the "nifi-content" node and add the "filename" node only using one JoltTransform JSON ; but i'm not very experienced using Jolt and I did not find out how.

Translate
Terms & ConditionsAccessibility statement