question Re: How to add custom attributes to AttributesToJSON? in Support Questions

How to add custom attributes to AttributesToJSON?

murali2425 — Wed, 23 Dec 2020 13:30:14 GMT

I have a scenario where list of files are coming from previous processor, where for each file, I have to create json file with attributes of the flowfile. In AttributesToJSON processor configuration there is option to extract pipeline attributes and can create json files/object, if we set Include Core Attributes to true, it will read some of the file properties and forms the json file.

the out for the above case in my scenario is …
{"fragment.size":"125"
file.group:"root",
file.lastModifiedTime:"2020-12-22T15:09:13+0000",
fragment.identifier:"ee5770ea-8406-400a-a2fd-2362bd706fe0",
fragment.index:"1",
file.creationTime:"2020-12-22T15:09:13+0000",
file.lastAccessTime:"2020-12-22T17:34:22+0000",
segment.original.filename:"Sample-Spreadsheet-10000-rows.csv",
file.owner:"root",
fragment.count:"2",
file.permissions:"rw-r--r--",
text.line.count:"1"}
}



 But the files has other properties, like absolute.path, filename, uuid  are missing in the above json file.
    My requirement is, get the absolute.path, filename and uuid and concatenate absolute.path+/+filename, assign this to custom attribute say filepath:absolute.path+/+filename and also add uuid to json object.

so my json file should like

{ uuid:"file uuid value", filepath:"absolute.path+/+filename" }

any inputs to get above form of json file.

Re: How to add custom attributes to AttributesToJSON?

MattWho — Wed, 23 Dec 2020 15:38:49 GMT

@murali2425

I was not able to reproduce the missing attributes in the content of a produced FlowFile from the AttributesToJson processor. What version of NiFi are you using? Did you inspect the attributes on the FlowFile in the immediate connection feeding the AttributesToJson before starting that processor?

Your desired output from the AttributesToJson seems to be very specific and not include all attributes including the core attributes anyway.

My suggestion would be to use an UpdateAttribute processor just before your AttributesToJson processor to build the specific attributes you want to have in your produced Json output content.

You would then add two custom dynamic properties where you would use NiFi Expression Language to populate the values from other attributes/metadata on the source FlowFile:

You could then configure your AttributesToJson processor to build the JSON content using only those two new attributes you just constructed:

Keep in mind that the AttributesToJson processor will add attributes to the Json in Lexicographical order. So if you want the uuid before the filepath, you will need to adjust the property names used in the UpdateAttribute processor.

For example
"auuid" instead of "myuuid" so that it comes before "filepath" in order.

Hope this helps,

Matt

Re: How to add custom attributes to AttributesToJSON?

murali2425 — Wed, 23 Dec 2020 17:33:37 GMT

Hi Matt,

Great, with your suggestion, I got what I was expecting.

Thank You,

--Murali