Is it the best practice to delete the user defined attributes after they have been used at the end of the NiFi flow?

[#1]

Hi, 

As  I learnt that the flow file attributes are held in memory in the nifi process, is it a good idea to clean up the attributes that have been defined earlier and delete them using UpdateAttribute processor if we know that we wouldn't use them anymore ? Or does deleting doesn't matter at all with regards to the memory usage? Please clarify.

[#2]

@Jayavardhini Yes, It is a best practice to auto-terminate successfully completed data flow flowfiles at the bottom of your flow branches.  You do not want them to remain as they will continue to hold resources.  

A good NiFi developer will build a flow during development where all bottom branches are visible, including routing all processor relationships, even ones that will eventually be auto terminated.   This gives visibility during testing and flow creation if someone unexpected happens.  I use stopped output ports for this purpose.  

In some of my production flows I create capture points for exceptions, these are bottom branch Process Groups or Queues where the flowfiles will remain until someone inspects them, makes a change, inspects the provenance, and maybe even reroutes it back into the flow again.   This is the only case where I keep flowfiles in my flow.  In all other cases I auto terminate and the flowfiles are gone from my flows.

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

Thanks,

Steven @ DFHZ

[#3]

Thanks Steven. But my question was specifically to the attributes that we define when needed in the flow and if the operation of deleting them at the end is particularly useful or not to improve the memory usage ?  So, i am just expecting an answer about the attributes here.

[#4 · Accepted solution]

Yes.  Auto-termination is how you drop the whole flowfile and all attributes.    

If for example, you have a giant attribute of SCHEMA or JSON,  using updateAttribute could be used to empty the value.   However if you do not even need the flowfile anymore, auto-terminate, versus updateAttribute and retain flowfile.

[#5]

Oh, that's great. Thanks for your response. That clarifies my question.