How to adapt the schema generated by inverAvroSchema

[#1]

Hello,

I use the processor inferavroschema to define the schema of my table SQL then I use PutDatabaseRecord except for some columns I have null values not detectable by inferavroschema For example for a column in my file csv it can have null values but so the schema avro they have string values. During the insertion I have errors I would like to know if there is a way to change the schema Avro without going through a manual writing of the structure of the table with Schema Registry.

Thank you

[#2]

Hi @Amira khalifa,

Why don't you want to define the schema or your target tables in the schema registry?

You can use InferAvroSchema to help you initializing the schema, then you can adapt it and copy/paste it into the schema registry of your choice. Using InferAvroSchema for every single flow file you'll ingest can be costly in terms of performance and it's only inferring the schema on a subset of data: as you can see, the schema might not be completely correct to take into account all the possibilities.

What you could do, even though it's not what I'd recommend as a long term solution, is to use expression language to update the avro schema generated as an attribute of your flow files.

Hope this helps.

[#3]

Hi @Pierre Villard

Thank you for your response.

I want to automate the work as much as possible. For a user of the application I want him to be able to specify only the file to import and then nifi does the necessary. I did a function that created a sql table automatically from inferAvroSchema and I want to adapt this schema. Avro Registry works manually. For each file I have to fill it manually.

[#4 · Accepted solution]

I used UpdateAttribut in which I get inferred.avro.schema with getAttribut and apply a 'replace' expression