Member since
05-09-2017
107
Posts
8
Kudos Received
6
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 4971 | 03-19-2020 01:30 PM | |
| 24200 | 11-27-2019 08:22 AM | |
| 11551 | 07-05-2019 08:21 AM |
03-19-2020
01:30 PM
I was able to resolve this issue after a lot of work. Records in kafka had null values and S3 sink connector cannot write null values to S3 bucket and failed with this error, We were able to dig deeper when we changed the flush.size =1 and then we saw a different error and that made us check for null values. We developed a patch that fixed the issue. now the S3 connector ignores null values . I dont know why confluent SMT did not work.
... View more
02-23-2020
10:56 AM
We recently upgraded Kafka to 2.3.0 after that one of our connector is failing with the below error. org.apache.kafka.connect.errors.ConnectException: org.apache.kafka.connect.errors.SchemaProjectorException: Switch between schema-based and schema-less data is not Any ideas anyone ? I have already looked and implemented this https://issues.redhat.com/browse/DBZ-235 but no luck. Nothing has changed in the schema. We dont think its a tombstone issue. I was able to check where the connector stopped and looked at the events and its not a tombstone.
... View more
Labels:
- Labels:
-
Apache Kafka
11-27-2019
10:25 AM
Now that the move has stopped/completed. Verify if there are no under-replicated OR offline partitions and also verify if there are 3 replicas and all are in sync (depends on your replication factor) You can delete those partitions manually and just restart the broker. If replicas are out of sync they must come back in sync if unclean.leader.election is true
... View more
11-27-2019
08:22 AM
Thats wierd. So in the log.dirs specify any,any,any and see if that works Example { "partitions": [{ "topic": "foo", "partition": 1, "replicas": [1, 2, 3], "log_dirs": ["any", "any", "any"] }], "version": 1 } This is what documentation says "Broker will cancel existing movement of the replica if "any" is specified as destination log directory."
... View more
11-26-2019
07:06 PM
Yes. What we are doing here is that we are stopping the reassignment that is not progressing. And to do that you run it with the original json file and it will stop and revert back.
... View more
11-26-2019
06:38 PM
To stop the move run the reassignment again with --execute option using the original {"version":1,"partitions":[{"topic":"XXXXX","partition":1,"replicas":[3,0,8],"log_dirs":["/data1/kafka","/data3/kafka","/data1/kafka"]}]} Then what you can do is instead of moving it on the same broker try to move the partition to a different broker
... View more
11-26-2019
06:17 PM
Do you have the original assignment ? You can stop it by passing the original assignment. It will just revert back. I did this couple of times and it works. Now regarding why its not completing can be an IO issue. Did you configure this value in the cluster num.replica.alter.log.dirs.threads. ? If not you might have to set it. Also might have to set intra.broker.throttled.rate using the safety valve. I read it in this KIP https://cwiki.apache.org/confluence/display/KAFKA/KIP-113%3A+Support+replicas+movement+between+log+directories Test in staging and then you can implement in prod.
... View more
07-05-2019
08:21 AM
1 Kudo
$ sudo tail -100 /var/log/cloudera-scm-server/db.log FATAL: could not create lock file "/var/run/postgresql/.s.PGSQL.7432.lock": No such file or directory $ sudo mkdir /var/run/postgresql $ sudo chown -R postgres:postgres /var/run/postgresql Once i did this it fixed the problem.
... View more
07-04-2019
07:44 PM
If itry to uninstall i get this and it wont allow me to re-install . so i rename that file to allow for new installation. sudo ./uninstall-cloudera-manager.sh ./uninstall-cloudera-manager.sh: line 2: /opt/cloudera/installer/.mojosetup/mojosetup: No such file or directory
... View more
07-04-2019
07:41 PM
Any idea what is mojosetup. Is it relevant here Jul 04 22:31:14 itlenscms01 mojosetup[2720]: FATAL: Installation failed. Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Cleaning up half-finished installation... Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/uninstall-cloudera-manager.sh' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-xenial/cloudera-manager-repository_6.2.0_all.deb' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-xenial' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-bionic/cloudera-manager-repository_6.2.0_all.deb' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-bionic' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/sles12/cloudera-manager-repository-6.2.0-1.noarch.rpm' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/sles12' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat7/cloudera-manager-repository-6.2.0-1.noarch.rpm' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat7' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat6/cloudera-manager-repository-6.2.0-1.noarch.rpm' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat6' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/debian-jessie/cloudera-manager-repository_6.2.0_all.deb' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/debian-jessie' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleting '/opt/cloudera/installer/.mojosetup/tmp/rollbacks' failed! Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleting '/opt/cloudera/installer/.mojosetup/tmp' failed! Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleting '/opt/cloudera/installer/.mojosetup' failed! Jul 04 22:31:16 itlenscms01 mojosetup[2720]: MojoSetup shutting down...
... View more