Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to add custom attributes to AttributesToJSON?

avatar
Rising Star

I have a scenario where list of files are coming from previous processor, where for each file, I have to create json file with attributes of the flowfile. In AttributesToJSON processor configuration there is option to extract pipeline attributes and can create json files/object, if we set Include Core Attributes to true, it will read some of the file properties and forms the json file.

 

AttributesToJSON.PNG

the out for the above case in my scenario is …
{"fragment.size":"125"
file.group:"root",
file.lastModifiedTime:"2020-12-22T15:09:13+0000",
fragment.identifier:"ee5770ea-8406-400a-a2fd-2362bd706fe0",
fragment.index:"1",
file.creationTime:"2020-12-22T15:09:13+0000",
file.lastAccessTime:"2020-12-22T17:34:22+0000",
segment.original.filename:"Sample-Spreadsheet-10000-rows.csv",
file.owner:"root",
fragment.count:"2",
file.permissions:"rw-r--r--",
text.line.count:"1"}
}



 But the files has other properties, like absolute.path, filename, uuid  are missing in the above json file.
    My requirement is, get the absolute.path, filename and uuid and concatenate absolute.path+/+filename, assign this to custom attribute say filepath:absolute.path+/+filename and also add uuid to json object.  

 

so my json file should like

{ uuid:"file uuid value", filepath:"absolute.path+/+filename" }

 

any inputs to get above  form of json file.

2 ACCEPTED SOLUTIONS

avatar
Master Mentor

@murali2425 

I was not able to reproduce the missing attributes in the content of a produced FlowFile from the AttributesToJson processor.  What version of NiFi are you using?  Did you inspect the attributes on the FlowFile in the immediate connection feeding the AttributesToJson before starting that processor?

Your desired output from the AttributesToJson seems to be very specific and not include all attributes including the core attributes anyway.

My suggestion would be to use an UpdateAttribute processor just before your AttributesToJson processor to build the specific attributes you want to have in your produced Json output content.

You would then add two custom dynamic properties where you would use NiFi Expression Language to populate the values from other attributes/metadata on the source FlowFile:

 

Screen Shot 2020-12-23 at 10.33.38 AM.png

 

You could then configure your AttributesToJson processor to build the JSON content using only those two new attributes you just constructed:

 

Screen Shot 2020-12-23 at 10.14.24 AM.png


Keep in mind that the AttributesToJson processor will add attributes to the Json in Lexicographical order. So if you want the uuid before the filepath, you will need to adjust the property names used in the UpdateAttribute processor.

For example
"auuid" instead of "myuuid" so that it comes before "filepath" in order.

Hope this helps,

Matt


 

View solution in original post

avatar
Rising Star

Hi Matt,

  Great,  with your suggestion, I got what I was expecting.

 

Thank You,

--Murali

View solution in original post

2 REPLIES 2

avatar
Master Mentor

@murali2425 

I was not able to reproduce the missing attributes in the content of a produced FlowFile from the AttributesToJson processor.  What version of NiFi are you using?  Did you inspect the attributes on the FlowFile in the immediate connection feeding the AttributesToJson before starting that processor?

Your desired output from the AttributesToJson seems to be very specific and not include all attributes including the core attributes anyway.

My suggestion would be to use an UpdateAttribute processor just before your AttributesToJson processor to build the specific attributes you want to have in your produced Json output content.

You would then add two custom dynamic properties where you would use NiFi Expression Language to populate the values from other attributes/metadata on the source FlowFile:

 

Screen Shot 2020-12-23 at 10.33.38 AM.png

 

You could then configure your AttributesToJson processor to build the JSON content using only those two new attributes you just constructed:

 

Screen Shot 2020-12-23 at 10.14.24 AM.png


Keep in mind that the AttributesToJson processor will add attributes to the Json in Lexicographical order. So if you want the uuid before the filepath, you will need to adjust the property names used in the UpdateAttribute processor.

For example
"auuid" instead of "myuuid" so that it comes before "filepath" in order.

Hope this helps,

Matt


 

avatar
Rising Star

Hi Matt,

  Great,  with your suggestion, I got what I was expecting.

 

Thank You,

--Murali