Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Pattern: convert CSV to JSON?

Solved Go to solution

Pattern: convert CSV to JSON?

Hi, what's the recommended processor sequence to parse single-line csv entries into JSON? I'm all set on ingest and egress, but a little fuzzy on the conversion part still.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Pattern: convert CSV to JSON?

If you have a fixed set of columns in the csv that you know ahead of time, you can use ExtractText + ReplaceText. This was an example I created once before, rename to .xml: csvtojson.txt

8 REPLIES 8

Re: Pattern: convert CSV to JSON?

How about - ConvertCSVToAvro - & then - ConvertAvroToJSON ?

Alternatively, You can use ExtractText with regex to convert CSV to JSON.

Re: Pattern: convert CSV to JSON?

If you have a fixed set of columns in the csv that you know ahead of time, you can use ExtractText + ReplaceText. This was an example I created once before, rename to .xml: csvtojson.txt

Highlighted

Re: Pattern: convert CSV to JSON?

Yes, it's a known CSV with a header line. I wonder if there's a trick to use the column names and expression language to avoid manual re-typing.

Re: Pattern: convert CSV to JSON?

There might be a cleaner way to do this, but if you had an incoming CSV like:

h1,h2,h3,h4

v1,v2,v3,v4

You could capture that in ExtractText with a pattern of:

(.+),(.+),(.+),(.+)\n(.+),(.+),(.+),(.+)

Then in ReplaceText:

{ "${csv.1}" : "${csv.5}", "${csv.2}" : "${csv.6}", "${csv.3}" : "${csv.7}", "${csv.4}" : "${csv.8}" }

Would produce:

{ "h1" : "v1", "h2" : "v2", "h3" : "v3", "h4" : "v4" }

Re: Pattern: convert CSV to JSON?

Guru

I would use ExtractText with Regex and then AttributestoJSON processor to create a JSON formated flow file.

Re: Pattern: convert CSV to JSON?

New Contributor

I have a convertCSVtoJSON processor. Will see about getting it contributed back.

Re: Pattern: convert CSV to JSON?

New Contributor

any word on your contribution ? I can use a csv2json processor/sequence now !!! :)

Re: Pattern: convert CSV to JSON?

In Apache NiFi 1.2.0 and 1.3.0 (HDF 3.0.0) there is a ConvertRecord processor that can convert between any combination of Avro, JSON, and CSV.