How to read json from S3 then edit json using Nifi

[#1]

I am using Nifi 1.6.0. I am trying to copy an S3 file into redshift. The json file on S3  looks like this:
[

{"a":1,"b":2},

{"a":3,"b":4}
]
However this gives an error because of '[' and ']' in the file (https://stackoverflow.com/a/45348425).

I need to convert the json from above format to a format which looks like this:

{"a":1,"b":2}

{"a":3,"b":4}

So basically what I am trying to do is to remove '[' and ']' and replace '},' with '}'.

The file has over 14 million rows (30 MB in size)

How can I achieve this using Nifi?

[#2]

@araujo 

I will try this solution, but I think the queue to PutDatabaseRecord processor will get clogged very soon because there are over 14 million rows in this file.

I tried Replace text and I was able to replace '},' to '}' within seconds. Can you tell me how to remove the first line '[' and the last line ']'?

Thanks

[#3 · Accepted solution]

@baymax277 ,

Try this:

This is the regex:

(?s)(^\[|\]$)

You may have to increase the buffer size to accommodate your file.

Cheers,

André