- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Kafka S3 sink connector failing
- Labels:
-
Apache Kafka
Created 02-23-2020 10:56 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We recently upgraded Kafka to 2.3.0 after that one of our connector is failing with the below error.
org.apache.kafka.connect.errors.ConnectException: org.apache.kafka.connect.errors.SchemaProjectorException: Switch between schema-based and schema-less data is not
Any ideas anyone ?
I have already looked and implemented this https://issues.redhat.com/browse/DBZ-235 but no luck.
Nothing has changed in the schema. We dont think its a tombstone issue. I was able to check where the connector stopped and looked at the events and its not a tombstone.
Created 03-19-2020 01:30 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I was able to resolve this issue after a lot of work.
Records in kafka had null values and S3 sink connector cannot write null values to S3 bucket and failed with this error, We were able to dig deeper when we changed the flush.size =1 and then we saw a different error and that made us check for null values.
We developed a patch that fixed the issue. now the S3 connector ignores null values . I dont know why confluent SMT did not work.
Created 03-19-2020 01:30 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I was able to resolve this issue after a lot of work.
Records in kafka had null values and S3 sink connector cannot write null values to S3 bucket and failed with this error, We were able to dig deeper when we changed the flush.size =1 and then we saw a different error and that made us check for null values.
We developed a patch that fixed the issue. now the S3 connector ignores null values . I dont know why confluent SMT did not work.
