question Re: Pattern: convert CSV to JSON? in Support Questions

Pattern: convert CSV to JSON?

andrewg — Thu, 15 Oct 2015 21:07:08 GMT

Hi, what's the recommended processor sequence to parse single-line csv entries into JSON? I'm all set on ingest and egress, but a little fuzzy on the conversion part still.

Re: Pattern: convert CSV to JSON?

bsaini — Thu, 15 Oct 2015 21:15:43 GMT

How about - ConvertCSVToAvro - & then - ConvertAvroToJSON ?

Alternatively, You can use ExtractText with regex to convert CSV to JSON.

Re: Pattern: convert CSV to JSON?

bbende — Thu, 15 Oct 2015 21:34:24 GMT

If you have a fixed set of columns in the csv that you know ahead of time, you can use ExtractText + ReplaceText. This was an example I created once before, rename to .xml: csvtojson.txt

Re: Pattern: convert CSV to JSON?

andrewg — Thu, 15 Oct 2015 21:42:16 GMT

Yes, it's a known CSV with a header line. I wonder if there's a trick to use the column names and expression language to avoid manual re-typing.

Re: Pattern: convert CSV to JSON?

bbende — Thu, 15 Oct 2015 22:53:39 GMT

There might be a cleaner way to do this, but if you had an incoming CSV like:

h1,h2,h3,h4

v1,v2,v3,v4

You could capture that in ExtractText with a pattern of:

(.+),(.+),(.+),(.+)\n(.+),(.+),(.+),(.+)

Then in ReplaceText:

{ "${csv.1}" : "${csv.5}", "${csv.2}" : "${csv.6}", "${csv.3}" : "${csv.7}", "${csv.4}" : "${csv.8}" }

Would produce:

{ "h1" : "v1", "h2" : "v2", "h3" : "v3", "h4" : "v4" }

Re: Pattern: convert CSV to JSON?

vvaks — Mon, 28 Mar 2016 07:05:33 GMT

I would use ExtractText with Regex and then AttributestoJSON processor to create a JSON formated flow file.

Re: Pattern: convert CSV to JSON?

randybovay — Sat, 10 Sep 2016 02:26:31 GMT

I have a convertCSVtoJSON processor. Will see about getting it contributed back.

Re: Pattern: convert CSV to JSON?

me2 — Fri, 07 Jul 2017 20:46:59 GMT

any word on your contribution ? I can use a csv2json processor/sequence now !!! 🙂

Re: Pattern: convert CSV to JSON?

bbende — Fri, 07 Jul 2017 21:32:37 GMT

In Apache NiFi 1.2.0 and 1.3.0 (HDF 3.0.0) there is a ConvertRecord processor that can convert between any combination of Avro, JSON, and CSV.