Do you have the original assignment ?  You can stop it by passing the original assignment. It will just revert back. I did this couple of times and it works.    Now regarding why its not completing can be an IO issue.  Did you configure this value in the cluster num.replica.alter.log.dirs.threads. ? If  not you might have to set it.  Also might have to set intra.broker.throttled.rate using the safety valve.    I read it in this KIP https://cwiki.apache.org/confluence/display/KAFKA/KIP-113%3A+Support+replicas+movement+between+log+directories    Test in staging and then you can implement in prod.  ... View more

If itry to uninstall i get this and it wont allow me to re-install . so i rename that file to allow for new installation.   sudo ./uninstall-cloudera-manager.sh ./uninstall-cloudera-manager.sh: line 2: /opt/cloudera/installer/.mojosetup/mojosetup: No such file or directory   ... View more

Any idea what is mojosetup. Is it relevant here   Jul 04 22:31:14 itlenscms01 mojosetup[2720]: FATAL: Installation failed. Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Cleaning up half-finished installation... Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/uninstall-cloudera-manager.sh' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-xenial/cloudera-manager-repository_6.2.0_all.deb' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-xenial' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-bionic/cloudera-manager-repository_6.2.0_all.deb' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-bionic' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/sles12/cloudera-manager-repository-6.2.0-1.noarch.rpm' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/sles12' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat7/cloudera-manager-repository-6.2.0-1.noarch.rpm' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat7' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat6/cloudera-manager-repository-6.2.0-1.noarch.rpm' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat6' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/debian-jessie/cloudera-manager-repository_6.2.0_all.deb' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/debian-jessie' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleting '/opt/cloudera/installer/.mojosetup/tmp/rollbacks' failed! Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleting '/opt/cloudera/installer/.mojosetup/tmp' failed! Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleting '/opt/cloudera/installer/.mojosetup' failed! Jul 04 22:31:16 itlenscms01 mojosetup[2720]: MojoSetup shutting down... ... View more

Still no luck.     I dont know why it says mysql. ?? how is this generated and from where ?   $ sudo cat /etc/cloudera-scm-server/db.properties.20190704-222918 # Copyright (c) 2012 Cloudera, Inc. All rights reserved. # # This file describes the database connection. # # The database type # Currently 'mysql', 'postgresql' and 'oracle' are valid databases. com.cloudera.cmf.db.type=mysql # The database host # If a non standard port is needed, use 'hostname:port' #com.cloudera.cmf.db.host=localhost # The database name #com.cloudera.cmf.db.name=cmf # The database user #com.cloudera.cmf.db.user=cmf # The database user's password #com.cloudera.cmf.db.password= # The db setup type # After fresh install it is set to INIT # and will be changed post config. # If scm-server uses Embedded DB then it is set to EMBEDDED # If scm-server uses External DB then it is set to EXTERNAL com.cloudera.cmf.db.setupType=INIT   And same error    $ sudo cat /var/log/cloudera-manager-installer/7.start-cloudera-scm-server-db.log Job for cloudera-scm-server-db.service failed because the control process exited with error code. See "systemctl status cloudera-scm-server-db.service" and "journalctl -xe" for details. $ sudo systemctl status cloudera-scm-server-db.service ● cloudera-scm-server-db.service - LSB: Cloudera SCM Server's Embedded DB Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server-db; bad; vendor preset: disabled) Active: failed (Result: exit-code) since Thu 2019-07-04 22:29:20 EDT; 4min 22s ago Docs: man:systemd-sysv-generator(8) Jul 04 22:29:19 itlenscms01 cloudera-scm-server-db[2984]: Creating SCM configuration file: /etc/cloudera-scm-server/db.properties Jul 04 22:29:20 itlenscms01 cloudera-scm-server-db[2984]: waiting for server to start.... stopped waiting Jul 04 22:29:20 itlenscms01 cloudera-scm-server-db[2984]: pg_ctl: could not start server Jul 04 22:29:20 itlenscms01 cloudera-scm-server-db[2984]: Examine the log output. Jul 04 22:29:20 itlenscms01 cloudera-scm-server-db[2984]: Could not start database server, giving up Jul 04 22:29:20 itlenscms01 runuser[2997]: pam_unix(runuser:session): session closed for user cloudera-scm Jul 04 22:29:20 itlenscms01 systemd[1]: cloudera-scm-server-db.service: control process exited, code=exited status=1 Jul 04 22:29:20 itlenscms01 systemd[1]: Failed to start LSB: Cloudera SCM Server's Embedded DB. Jul 04 22:29:20 itlenscms01 systemd[1]: Unit cloudera-scm-server-db.service entered failed state. Jul 04 22:29:20 itlenscms01 systemd[1]: cloudera-scm-server-db.service failed. Warning: cloudera-scm-server-db.service changed on disk. Run 'systemctl daemon-reload' to reload units. ... View more

I just retried it without the --throttle option and it works fine and i was able to sucessfully move two partitions across volumes in same broker. Summary is to use it without throttling.    I initially tried with throttle and realized it unable to complete the move as the messages are also coming in at a steady pace. i also tried increasing and removing the throttle completely and it did not complete the move. Maybe something is going on with throttle.   ... View more

Its been quitesome time that i ran that and dont have the session details. However i think the output would say "reassingment of replica still in progress" .   Reassignment of partition prod-events-45 completed successfully Reassignment of replica prod-events-45-98 in progress Reassignment of replica prod-events-45-154 completed successfully Reassignment of replica prod-events-45-157 completed successfully   I tested this in staging cluster when there are no incoming messages, it works fine. In production this topic events were being written at 3000 events/sec and the move did not complete. I was able to somehow stop the reassingment without causing any issues to paritions  ... View more

I am using kafka 2.0    I am moving a partition that is around 76G  . You can see the size from my original post.    kafka-reassign-partitions --zookeeper xxxx:2181/kafka --execute --reassignment-json-file reassign.json --bootstrap-server xxxx:9092 ... View more