Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV

Solved Go to solution
Highlighted

Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV

New Contributor

I am unable to find a processor which can convert Avro to CSV or Json to CSV .. any pointer on this ..

,

I am working on a data flow, where I am reading data from Redshift and then pushing data to a .csv file. All is good, however I am unable to find a processor which can convert data to CSV format.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV

Super Guru

@Faisal Hussain please take a look at this post.

@Bryan Bende mentioned this:

incoming CSV like:

h1,h2,h3,h4

v1,v2,v3,v4

You could capture that in ExtractText with a pattern of:

(.+),(.+),(.+),(.+)\n(.+),(.+),(.+),(.+)

Then in ReplaceText:

{ "${csv.1}" : "${csv.5}", "${csv.2}" : "${csv.6}", "${csv.3}" : "${csv.7}", "${csv.4}" : "${csv.8}" }

Would produce:

{ "h1" : "v1", "h2" : "v2", "h3" : "v3", "h4" : "v4" }

3 REPLIES 3

Re: Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV

Super Guru

@Faisal Hussain please take a look at this post.

@Bryan Bende mentioned this:

incoming CSV like:

h1,h2,h3,h4

v1,v2,v3,v4

You could capture that in ExtractText with a pattern of:

(.+),(.+),(.+),(.+)\n(.+),(.+),(.+),(.+)

Then in ReplaceText:

{ "${csv.1}" : "${csv.5}", "${csv.2}" : "${csv.6}", "${csv.3}" : "${csv.7}", "${csv.4}" : "${csv.8}" }

Would produce:

{ "h1" : "v1", "h2" : "v2", "h3" : "v3", "h4" : "v4" }

Re: Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV

Super Guru

Re: Nifi - AVRO to CSV or Json to CSV,NIFI - convert Avro to CSV

@Faisal Hussain I do not think there is one but definitely there is plan to develop one.

https://cwiki.apache.org/confluence/display/NIFI/First-class+Avro+Support

You can convert Avro to JSON (ConvertAvroToJSON) and then call a script (lots of example of scripts in the internet for converting JSON to csv using java/javascript/perl/bash/awk etc and etc) in the ExecuteStreamCommand processor. This would work. Let us know if you need more details.