Member since
05-09-2017
107
Posts
7
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3175 | 03-19-2020 01:30 PM | |
16317 | 11-27-2019 08:22 AM | |
8825 | 07-05-2019 08:21 AM | |
15428 | 09-25-2018 12:09 PM | |
5839 | 08-10-2018 07:46 AM |
11-26-2019
06:17 PM
Do you have the original assignment ? You can stop it by passing the original assignment. It will just revert back. I did this couple of times and it works. Now regarding why its not completing can be an IO issue. Did you configure this value in the cluster num.replica.alter.log.dirs.threads. ? If not you might have to set it. Also might have to set intra.broker.throttled.rate using the safety valve. I read it in this KIP https://cwiki.apache.org/confluence/display/KAFKA/KIP-113%3A+Support+replicas+movement+between+log+directories Test in staging and then you can implement in prod.
... View more
07-05-2019
08:21 AM
$ sudo tail -100 /var/log/cloudera-scm-server/db.log FATAL: could not create lock file "/var/run/postgresql/.s.PGSQL.7432.lock": No such file or directory $ sudo mkdir /var/run/postgresql $ sudo chown -R postgres:postgres /var/run/postgresql Once i did this it fixed the problem.
... View more
07-04-2019
07:44 PM
If itry to uninstall i get this and it wont allow me to re-install . so i rename that file to allow for new installation. sudo ./uninstall-cloudera-manager.sh ./uninstall-cloudera-manager.sh: line 2: /opt/cloudera/installer/.mojosetup/mojosetup: No such file or directory
... View more
07-04-2019
07:41 PM
Any idea what is mojosetup. Is it relevant here Jul 04 22:31:14 itlenscms01 mojosetup[2720]: FATAL: Installation failed. Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Cleaning up half-finished installation... Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/uninstall-cloudera-manager.sh' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-xenial/cloudera-manager-repository_6.2.0_all.deb' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-xenial' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-bionic/cloudera-manager-repository_6.2.0_all.deb' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/ubuntu-bionic' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/sles12/cloudera-manager-repository-6.2.0-1.noarch.rpm' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/sles12' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat7/cloudera-manager-repository-6.2.0-1.noarch.rpm' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat7' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat6/cloudera-manager-repository-6.2.0-1.noarch.rpm' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/redhat6' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/debian-jessie/cloudera-manager-repository_6.2.0_all.deb' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages/debian-jessie' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleted '/opt/cloudera/installer/packages' Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleting '/opt/cloudera/installer/.mojosetup/tmp/rollbacks' failed! Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleting '/opt/cloudera/installer/.mojosetup/tmp' failed! Jul 04 22:31:16 itlenscms01 mojosetup[2720]: Deleting '/opt/cloudera/installer/.mojosetup' failed! Jul 04 22:31:16 itlenscms01 mojosetup[2720]: MojoSetup shutting down...
... View more
07-04-2019
07:31 PM
Still no luck. I dont know why it says mysql. ?? how is this generated and from where ? $ sudo cat /etc/cloudera-scm-server/db.properties.20190704-222918 # Copyright (c) 2012 Cloudera, Inc. All rights reserved. # # This file describes the database connection. # # The database type # Currently 'mysql', 'postgresql' and 'oracle' are valid databases. com.cloudera.cmf.db.type=mysql # The database host # If a non standard port is needed, use 'hostname:port' #com.cloudera.cmf.db.host=localhost # The database name #com.cloudera.cmf.db.name=cmf # The database user #com.cloudera.cmf.db.user=cmf # The database user's password #com.cloudera.cmf.db.password= # The db setup type # After fresh install it is set to INIT # and will be changed post config. # If scm-server uses Embedded DB then it is set to EMBEDDED # If scm-server uses External DB then it is set to EXTERNAL com.cloudera.cmf.db.setupType=INIT And same error $ sudo cat /var/log/cloudera-manager-installer/7.start-cloudera-scm-server-db.log Job for cloudera-scm-server-db.service failed because the control process exited with error code. See "systemctl status cloudera-scm-server-db.service" and "journalctl -xe" for details. $ sudo systemctl status cloudera-scm-server-db.service ● cloudera-scm-server-db.service - LSB: Cloudera SCM Server's Embedded DB Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server-db; bad; vendor preset: disabled) Active: failed (Result: exit-code) since Thu 2019-07-04 22:29:20 EDT; 4min 22s ago Docs: man:systemd-sysv-generator(8) Jul 04 22:29:19 itlenscms01 cloudera-scm-server-db[2984]: Creating SCM configuration file: /etc/cloudera-scm-server/db.properties Jul 04 22:29:20 itlenscms01 cloudera-scm-server-db[2984]: waiting for server to start.... stopped waiting Jul 04 22:29:20 itlenscms01 cloudera-scm-server-db[2984]: pg_ctl: could not start server Jul 04 22:29:20 itlenscms01 cloudera-scm-server-db[2984]: Examine the log output. Jul 04 22:29:20 itlenscms01 cloudera-scm-server-db[2984]: Could not start database server, giving up Jul 04 22:29:20 itlenscms01 runuser[2997]: pam_unix(runuser:session): session closed for user cloudera-scm Jul 04 22:29:20 itlenscms01 systemd[1]: cloudera-scm-server-db.service: control process exited, code=exited status=1 Jul 04 22:29:20 itlenscms01 systemd[1]: Failed to start LSB: Cloudera SCM Server's Embedded DB. Jul 04 22:29:20 itlenscms01 systemd[1]: Unit cloudera-scm-server-db.service entered failed state. Jul 04 22:29:20 itlenscms01 systemd[1]: cloudera-scm-server-db.service failed. Warning: cloudera-scm-server-db.service changed on disk. Run 'systemctl daemon-reload' to reload units.
... View more
07-04-2019
06:19 PM
# The database type # Currently 'mysql', 'postgresql' and 'oracle' are valid databases. com.cloudera.cmf.db.type=mysql How can that be changed to postgres. I tried so many things and now out of options. I am doing a POC and at first started the regular installation bu installing agent,server.daemons and bringing up CM server without a DB and then unstalled all retried using the cloudera-manager-installer.bin . After a day of searching i cannot get the installtion to complete. The embedded database does not install with the below error $$$$sudo cat /var/log/cloudera-manager-installer/7.start-cloudera-scm-server-db.log Job for cloudera-scm-server-db.service failed because the control process exited with error code. See "systemctl status cloudera-scm-server-db.service" and "journalctl -xe" for details. $$$$ systemctl status cloudera-scm-server-db.service ● cloudera-scm-server-db.service - LSB: Cloudera SCM Server's Embedded DB Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server-db; bad; vendor preset: disabled) Active: failed (Result: exit-code) since Thu 2019-07-04 21:09:39 EDT; 6min ago Docs: man:systemd-sysv-generator(8) Warning: cloudera-scm-server-db.service changed on disk. Run 'systemctl daemon-reload' to reload units. ## Tried systemctl daemon-reload many times still no luck and then reran installer @Harsh J , @lhebert any thoughts here.
... View more
Labels:
- Labels:
-
Cloudera Manager
06-17-2019
12:04 PM
I just retried it without the --throttle option and it works fine and i was able to sucessfully move two partitions across volumes in same broker. Summary is to use it without throttling. I initially tried with throttle and realized it unable to complete the move as the messages are also coming in at a steady pace. i also tried increasing and removing the throttle completely and it did not complete the move. Maybe something is going on with throttle.
... View more
06-05-2019
05:52 PM
Its been quitesome time that i ran that and dont have the session details. However i think the output would say "reassingment of replica still in progress" . Reassignment of partition prod-events-45 completed successfully Reassignment of replica prod-events-45-98 in progress Reassignment of replica prod-events-45-154 completed successfully Reassignment of replica prod-events-45-157 completed successfully I tested this in staging cluster when there are no incoming messages, it works fine. In production this topic events were being written at 3000 events/sec and the move did not complete. I was able to somehow stop the reassingment without causing any issues to paritions
... View more
06-05-2019
09:57 AM
I am using kafka 2.0 I am moving a partition that is around 76G . You can see the size from my original post. kafka-reassign-partitions --zookeeper xxxx:2181/kafka --execute --reassignment-json-file reassign.json --bootstrap-server xxxx:9092
... View more
05-22-2019
04:51 PM
This is a partition move from one volume to another in a single broker. I have started a kafka-reaasign-partitions of a partition from one volume to another but it never completes. Has anyone faced this ? Kafka cluster is in a healthy state Controller is fine source and destination is also fine The replica is not a leader. So far havent seen any errors in logs The directories look like the below. Here i am moving a partition from sdc to sdf volume. inodes also look fine. desind@xxx:~#> sudo du -sh /kafka/data/sdc/prod-events-48 76G /kafka/data/sdc/prod-events-48 desind@xxx:~#> sudo du -sh /kafka/data/sdf/prod-events-48.b82f63b489554ef4b2e2f4c514bc1bc0-future 76G /kafka/data/sdf/prod-events-48.b82f63b489554ef4b2e2f4c514bc1bc0-future desind@xxx:~#> find /kafka/data -type d -name 'prod-events-48*' /kafka/data/sdf/prod-events-48.b82f63b489554ef4b2e2f4c514bc1bc0-future /kafka/data/sdc/prod-events-48
... View more
Labels:
- Labels:
-
Apache Kafka