Can't redefine org.apache.nifi.dataType - Schema error

[#1]

Hi Guys,

I'm currently having an issue where a schema that I'm using is erroring out in a RecordLookup processor ( with a Rest Service ). The message I get is the one in the title of this post, "Can't redefine org.apache.nifi.dataType". Below you can find the schema that I'm using.

{
  "type": "record",
  "name": "FullTicketData",
  "fields": [
    {
      "name" : "terminal",
      "type": {
        "type": "record",
        "name" : "terminal_array",
        "fields": [
          {
            "name": "success",
            "type": "string"
          },
          {
            "name": "data",
            "type": {
              "type": "array",
              "items": {
                "name": "data_array",
                "type": "record",
                "fields": [
                  {
                    "name": "ID",
                    "type": "int"
                  }
                ]
              }
            }
          },
          {
            "name": "count",
            "type": "int"
          }
        ]
      }
    },
    {
      "name" : "driver",
      "type": {
        "type": "record",
        "name" : "driver",
        "fields": [
          {
            "name": "success",
            "type": "string"
          },
          {
            "name": "data",
            "type": {
              "type": "array",
              "items": {
                "name": "data_record",
                "type": "record",
                "fields": [
                  {
                    "name": "ID",
                    "type": "int"
                  }
                ]
              }
            }
          },
          {
            "name": "count",
            "type": "int"
          }
        ]
      }
    },
    {
      "name": "ticket",
      "type": {
        "name" : "ticket",
        "type" : "record",
        "fields": [
          {
            "name": "success",
            "type": "string"
          },
          {
            "name": "data",
            "type": {
              "name": "data",
              "type": "record",
              "fields": [
                {
                  "name": "ID",
                  "type": "int"
                },
                {
                  "name": "job",
                  "type": {
                    "name": "job",
                    "type": "record",
                    "fields": [
                      {
                        "name": "ID",
                        "type": "int"
                      }
                    ]
                  }
                }
              ]
            }
          }
        ]
      }
    },
    {
      "name": "order",
      "type": {
        "name": "order",
        "type": "record",
        "fields": [
          {
            "name": "externalJobNo",
            "type": "string"
          },
          {
            "name": "jobNo",
            "type": "string"
          },
          {
            "name": "driverName",
            "type": "string"
          },
          {
            "name": "startTime",
            "type": "string"
          },
          {
            "name": "endTime",
            "type": "string"
          },
          {
            "name": "comment",
            "type": "string",
            "default": "empty"
          },
          {
            "name": "readStart",
            "type": "string",
            "default": "0"
          },
          {
            "name": "readEnd",
            "type": "string",
            "default": "0"
          },
          {
            "name": "removed",
            "type": "string"
          },
          {
            "name": "installed",
            "type": "string"
          },
          {
            "name": "endTopCut",
            "type": "string"
          },
          {
            "name": "endBottomCut",
            "type": "string"
          },
          {
            "name": "startTemp",
            "type": "string"
          },
          {
            "name": "endTemp",
            "type": "string"
          },
          {
            "name": "swPer",
            "type": "string"
          },
          {
            "name": "gravity",
            "type": "string"
          },
          {
            "name": "volume",
            "type": "string"
          },
          {
            "name": "grossVolume",
            "type": "string"
          },
          {
            "name": "temp",
            "type": "string"
          },
          {
            "name": "dist",
            "type": "string"
          },
          {
            "name": "explanation",
            "type": "string"
          },
          {
            "name": "status",
            "type": "string"
          },
          {
            "name": "driverID",
            "type": "int"
          },
          {
            "name": "billingQuestionComment",
            "type": "string",
            "default": "empty"
          }
        ]
      }
    },
    {
      "name": "jobTimeType",
      "type": {
        "name": "jobTimeType",
        "type": "record",
        "fields": [
          {
            "name": "startTime",
            "type": "string"
          },
          {
            "name": "endTime",
            "type": "string"
          },
          {
            "name": "startlatitude",
            "type": "string"
          },
          {
            "name": "startlongitude",
            "type": "string"
          },
          {
            "name": "endlatitude",
            "type": "string"
          },
          {
            "name": "endlongitude",
            "type": "string"
          }
        ]
      }
    },
    {
      "name": "jobTimeTypeTerminal",
      "type": {
        "name": "jobTimeTypeTerminal",
        "type": "record",
        "fields": [
          {
            "name": "terminal",
            "type": "string"
          },
          {
            "name": "startTime",
            "type": "string"
          },
          {
            "name": "endTime",
            "type": "string"
          },
          {
            "name": "startlatitude",
            "type": "string"
          },
          {
            "name": "startlongitude",
            "type": "string"
          },
          {
            "name": "endlatitude",
            "type": "string"
          },
          {
            "name": "endlongitude",
            "type": "string"
          }
        ]
      }
    }
  ]
}

And the data, that it maps. BTW, this is just an example data.

{
    "terminal": {
        "success": "string",
        "data": [
            {
                "ID": 0
            }
        ],
        "count": 0
    },
    "driver": {
        "success": "string",
        "data": [
            {
                "ID": 0
            }
        ],
        "count": 0
    },
    "ticket": {
        "success": "string",
        "data": {
            "ID": 0,
            "job": {
                "ID": 0
            }
        }
    },
    "order": {
        "externalJobNo": "string",
        "jobNo": "string",
        "driverName": "string",
        "startTime": "string",
        "endTime": "string",
        "comment": "string",
        "readStart": "string",
        "readEnd": "string",
        "removed": "string",
        "installed": "string",
        "endTopCut": "string",
        "endBottomCut": "string",
        "startTemp": "string",
        "endTemp": "string",
        "swPer": "string",
        "gravity": "string",
        "volume": "string",
        "grossVolume": "string",
        "temp": "string",
        "dist": "string",
        "explanation": "string",
        "status": "string",
        "driverID": 0,
        "billingQuestionComment": "string"
    },
    "jobTimeType": {
        "startTime": "string",
        "endTime": "string",
        "startlatitude": "string",
        "startlongitude": "string",
        "endlatitude": "string",
        "endlongitude": "string"
    },
    "jobTimeTypeTerminal": {
        "terminal": "string",
        "startTime": "string",
        "endTime": "string",
        "startlatitude": "string",
        "startlongitude": "string",
        "endlatitude": "string",
        "endlongitude": "string"
    }
}

So, for testing, I tried to use the GenerateFlowFile processor, with the exact sample data above, and a ConvertRecord processor set to Infer Schema, to make NiFi generate the schema so I can spot any errors, but when I do that, I get the same error message. 

The last piece of information I have is that this error started when I added the "terminal" and the "driver" JSON attributes. Without it, everything seems to work correctly. 

[#2]

@hjfigueira ,

I ran a test with your schema and data and it worked without problems for me.

I think there's something else in your environment that hasn't been described here that's contributing to the problem.

André

[#3]

Thank you very much @araujo.

Finally, I manage to solve that, I'm just unsure about the true cause tho. What I did was, on the service that writes the JSON, in the "Schema Access Strategy" instead of using "Inherit Schema" I used the "Schema Text" property, and used that same schema. ( I already had it into the reader ).

I suspect that somehow, the schemas that NiFi was generating, contained duplicated data, but I have no hard evidence on that so, it's just speculation. 

[#4]

Which version of NiFi are you using?

[#5]

1.13.2

[#6]

Thanks!