Json to Avro conversion using Confluent Schema Registry

[#1]

Hi, 

I want to convert the data from JSON to Avro using the ConvertRecord processor. This is my flow and the error I am getting:

I have a docker image for the confluent schema registry. The curl for registering the schema in the schema registry is :

curl --location --request GET 'http://localhost:8085/schemas/ids/42' \

--header 'Content-Type: application/vnd.schemaregistry.v1+json' \

--data '{"schema": "{\"type\":\"record\",\"name\":\"User\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"username\",\"type\":\"string\"},{\"name\":\"email\",\"type\":\"string\"}]}"}'

Generate flow file configurations:

convertRecord Configurations:

AvroRecordSetWriter configurations:

JsonTreeReader configurations:

Confluent Schema Registry :

[#2 · Accepted solution]

hi @nisha2112,

Are you certain that your schema is correct? I do not have to much experience with the ConfluentSchemaRegistry, but I think that you might have altered your schema, either when inserting it into the registry or when exporting it out of the registry.

What I recommend you to do is:

- retrieve the schema (AS-IS) and check it to see if it is correct or not. If not, you know what to do. If it is correct, proceed to next point.
- Within your ConvertRecord, modify both your Reader and your Writer to use Schema Text Property, where you manually define your schema. This will tell you one of the following two things:

1) Either your data coming into ConvertRecord is not in a correct format. --> ConvertRecord will fail.
2) You schema gets extracted fault from your ConfluentSchemaRegistry. --> The Flow will work and you will have no error.

Once you did the test, you will know where the error is located and you can try debugging it further. For example, you can try and extract your schema from ConfluentSchemaRegistry and see if it gets extracted accordingly. Or if your data is incorrect, you can check if something changed in your source and you modify that data or your schema. There are plenty of possibilities and you have to start from somewhere 🙂