I am retrieving a csv content from a JDBC connection using ExecuteSQLRecord processor and I would like to add to the attribute the schema information id.
I can easily set an attribute with the avro schema definition of the query but I would like that NiFi wrote the schema to the Confluent Registry, retrieved the schema Id and add it to the flow attribute.
To achieve this is I set the following properties in the ExecuteSQLRecord:
Database Connection Pooling Service
No value set
SQL select query
No value set
Max Wait Time
Normalize Table/Column Names
Use Avro Logical Types
Max Rows Per Flow File
Output Batch Size
In the CSVRecordSetWriter I have the following settings:
|Schema Write Strategy||Confluent Schema Registry Reference|
|Schema Cache||No value set|
|Schema Access Strategy||Inherit Record Schema|
And in the ConfluentSchemaRegistry I have:
|Schema Registry URLs||http://host.docker.internal:8081|
|SSL Context Service||No value set|
|Communications Timeout||30 secs|
|Cache Expiration||1 hour|
But, when I try to run it I get the following error:
org.apache.nifi.schema.access.SchemaNotFoundException: Cannot write Confluent Schema registry reference because the schema identifier is not known.
So I have two questions:
Thanks in advance,
While we welcome your question and a member of the Cloudera Community might well be able to answer it, I believe you would be much more likely to obtain a timely solution if you posted it to the Confluent Community.