Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

base64 encoded file to JSON string

avatar
New Contributor

Hi,

I'm building a JSON with AttributestoJSON that I need to send to a REST service, but I'm having trouble to figure out how to add to this JSON a file that I get from FetchFTP, then I encode it with base64encodeContent processor.

Any idea?

Thank you very much indeed.

1 ACCEPTED SOLUTION

avatar
Master Guru

What do you mean by "add to this JSON a file that I get from FetchFTP"? Is the file you're fetching a JSON file, and you want to add fields to it? Are you Base64 encoding just the JSON from the attributes or the entire file after adding to it?

If the incoming file (from FTP) is JSON, and you can get your attributes added to that flow file, then (as of NiFi 1.2.0 / HDF 3.0) you can use JoltTransformJSON to inject your individual attributes as fields into your JSON document (instead of AttributesToJSON). If you have too many attributes for that, your options are a bit more limited. In NiFi 1.3.0, you can use UpdateRecord to add the JSON from an attribute into a field in the other JSON document. You can also do this manually with ReplaceText. However one of the two JSON objects must be in an attribute. Whichever of the two (from AttributesToJSON or FetchFTP) is smaller, you can get that object first and use ExtractText to put the whole thing into an attribute. Note that attributes have limited size and introduce more memory usage, so beware of large JSON objects in attributes. However if one of them fits in a attribute, then you can use the UpdateRecord or ReplaceText processor as described.

If you need to just encode one of the JSON objects, then if it is in an attribute you can use UpdateAttribute with the base64Encode Expression Language function, or if it is in content you can use the Base64EncodeContent processor.

View solution in original post

2 REPLIES 2

avatar
Master Guru

What do you mean by "add to this JSON a file that I get from FetchFTP"? Is the file you're fetching a JSON file, and you want to add fields to it? Are you Base64 encoding just the JSON from the attributes or the entire file after adding to it?

If the incoming file (from FTP) is JSON, and you can get your attributes added to that flow file, then (as of NiFi 1.2.0 / HDF 3.0) you can use JoltTransformJSON to inject your individual attributes as fields into your JSON document (instead of AttributesToJSON). If you have too many attributes for that, your options are a bit more limited. In NiFi 1.3.0, you can use UpdateRecord to add the JSON from an attribute into a field in the other JSON document. You can also do this manually with ReplaceText. However one of the two JSON objects must be in an attribute. Whichever of the two (from AttributesToJSON or FetchFTP) is smaller, you can get that object first and use ExtractText to put the whole thing into an attribute. Note that attributes have limited size and introduce more memory usage, so beware of large JSON objects in attributes. However if one of them fits in a attribute, then you can use the UpdateRecord or ReplaceText processor as described.

If you need to just encode one of the JSON objects, then if it is in an attribute you can use UpdateAttribute with the base64Encode Expression Language function, or if it is in content you can use the Base64EncodeContent processor.

avatar
New Contributor

Sorry, yes I solved it before reading your answer Matt, but thank you very much indeed. OK, it's done, using ExtractText Processor with DOTALL mode to "true" and (.*) as regex and then to AttributesToJSON processor with the name given to the regex in the AttributesList.