question Re: Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV in Archives of Support Questions (Read Only)

Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV

fhussain — Fri, 16 Sep 2022 10:28:54 GMT

I am unable to find a processor which can convert Avro to CSV or Json to CSV .. any pointer on this ..

I am working on a data flow, where I am reading data from Redshift and then pushing data to a .csv file. All is good, however I am unable to find a processor which can convert data to CSV format.

Re: Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV

sunile_manjee — Sat, 09 Jul 2016 00:11:55 GMT

@Faisal Hussain please take a look at this post.

@Bryan Bende mentioned this:

incoming CSV like:

h1,h2,h3,h4

v1,v2,v3,v4

You could capture that in ExtractText with a pattern of:

(.+),(.+),(.+),(.+)\n(.+),(.+),(.+),(.+)

Then in ReplaceText:

{ "${csv.1}" : "${csv.5}", "${csv.2}" : "${csv.6}", "${csv.3}" : "${csv.7}", "${csv.4}" : "${csv.8}" }

Would produce:

{ "h1" : "v1", "h2" : "v2", "h3" : "v3", "h4" : "v4" }

Re: Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV

TimothySpann — Sat, 09 Jul 2016 00:18:29 GMT

https://nifi.apache.org/docs.html

see this article

https://community.hortonworks.com/articles/28341/converting-csv-to-avro-with-apache-nifi.html

Re: Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV

rbiswas1 — Sat, 09 Jul 2016 00:39:36 GMT

@Faisal Hussain I do not think there is one but definitely there is plan to develop one.

https://cwiki.apache.org/confluence/display/NIFI/First-class+Avro+Support

You can convert Avro to JSON (ConvertAvroToJSON) and then call a script (lots of example of scripts in the internet for converting JSON to csv using java/javascript/perl/bash/awk etc and etc) in the ExecuteStreamCommand processor. This would work. Let us know if you need more details.