Support Questions
Find answers, ask questions, and share your expertise

Convert JSON to CSV with Map

Convert JSON to CSV with Map

New Contributor

I read a lot about convertig json to csv or the other way around. But somehow it seems like I'm still doing something wrong. My JSON, which I get, looks like this:

{
  "YLMMR" : [ {
    "stunde" : "2019-01-22 13:00:00",
    "bstid" : "YLMMR",
    "height" : "372.00",
    "temperature" : "-5.80",
    "wind" : "10.080",
    "gusts" : "17.280",
    "snowfall" : "0.00",
    "rain" : "0.00",
    "snowheight" : "0.70"
  },
  {
    "stunde" : "2019-01-22 14:00:00",
    "bstid" : "YLMMR",
    "height" : "386.40",
    "temperature" : "-1.80",
    "wind" : "6.120",
    "gusts" : "29.880",
    "snowfall" : "0.00",
    "rain" : "0.00",
    "snowheight" : "1.00"
  }],
  "YLLCI" : [ {
    "stunde" : "2019-01-22 13:00:00",
    "bstid" : "YLLCI",
    "height" : "386.40",
    "temperature" : "-1.80",
    "wind" : "6.120",
    "gusts" : "29.880",
    "snowfall" : "0.00",
    "rain" : "0.00",
    "snowheight" : "1.00"
  },
  {
    "stunde" : "2019-01-22 14:00:00",
    "bstid" : "YLLCI",
    "height" : "386.40",
    "temperature" : "-1.80",
    "wind" : "6.120",
    "gusts" : "29.880",
    "snowfall" : "0.00",
    "rain" : "0.00",
    "snowheight" : "1.00"
  } ]
}

YLMMR and YLLCI are only two of many values which are all different. So I thought I need my AvroSchema to define a record with map inside and inside the map records. So I defined my schema like this:

{
  "name": "wetter",
  "type": "record",
  "fields": [{
        "name": "bst",
        "type": {
              "type": "map",
              "values": {
                "type": "array",
                "items": {
                  "name": "datum",
                  "type": "record",
                  "fields": [
                    {"name": "stunde", "type": "string"},
                    {"name": "bstid", "type": "string"},
                    {"name": "height", "type": "string"},
                    {"name": "temperature", "type": "string"},
                    {"name": "wind", "type": "string"},
                    {"name": "gusts", "type": "string"},
                    {"name": "snowfall", "type": "string"},
                    {"name": "rain", "type": "string"},
                    {"name": "snowheight", "type": "string"}
                  ]
                }
              }
            }
        }
    ]
}

My Nifi-job looks like the attachment "nifi-job.png"

98382-nifi-job.png

And I have a JsonTreeReader as well as a CSVRecordSetWriter. If any more information are needed, then please let me know. The file that comes out of the convertion processor contains just "bst" and nothing else.

Best regards
Marco