Manage Kudu from command line when CM is down

[#1]

Hello,

i'm using a small 3Master+3workers cluster [Cloudera Manager 7.3.1 (Runtime 7.1.6)]

Postgresql HA configured (not embedded)

The thing is that I need to restart a tablet server if hte CM host is down, and obviously I can't do it using the CM.

is there any way to restart these Kudu tablet servers without CM?

NOTE: I've already tried sudo service kudu-tserver stop

but

Failed to stop kudu-tserver.service: Unit kudu-tserver.service not loaded.

[#2]

Hi @Juanes,

The command you shared should work, given that kudu-tserver process is actually running on the host. Can you list all processes and grep for "kudu" on that host? 

For more information see this documentation page on starting kudu on the host. 

If this helps, please accept as a solution.

Regards,

Alex

[#3]

Good morning and thanks for your help,

I already read this page but the command wasn't loaded as systemctl proccess:

# sudo service kudu-tserver restart
Redirecting to /bin/systemctl restart kudu-tserver.service
Failed to restart kudu-tserver.service: Unit not found.

I checked the kudu-tablet proccess and it was executed from the /opt/parcel:

kudu 30973 30965 99 07:47 ? 00:05:51 /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/kudu/sbin/kudu-tserver --tserver_master_addrs=master1,master2,master3 --flagfile=/var/run/cloudera-scm-agent/process/1546342994-kudu-KUDU_TSERVER/gflagfile

I tried as well executing this kudu-tserver but it doesn't work:

# sudo /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/kudu/sbin/kudu-tserver stop
usage: /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/kudu/sbin/kudu-tserver

[#4]

Hello,

This command is not working, the binaries are running under the parcel installation and they are not loaded in the systemctl.

I've tried to execute the binaries from /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/kudu/sbin/kudu-tserver but it doesn't work properly.

root@server:~# sudo /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/kudu/sbin/kudu-tserver --help
kudu-tserver: Warning: SetUsageMessage() never called

Flags from ../../../../../src/kudu/cfile/block_cache.cc:
-block_cache_capacity_mb (block cache capacity in MB) type: int64
default: 512
-block_cache_type (Which type of block cache to use for caching data. Valid
choices are 'DRAM' or 'NVM'. DRAM, the default, caches data in regular
memory. 'NVM' caches data in a memory-mapped file using the memkind
library. To use 'NVM', libmemkind 1.8.0 or newer must be available on the
system; otherwise Kudu will crash.) type: string default: "DRAM"
-force_block_cache_capacity (Force Kudu to accept the block cache size,
even if it is unsafe.) type: bool default: false [...]

[#5 · Accepted solution]

I'm finally using the binaries in the parcel dir:

# sudo -u kudu /opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/bin/kudu master list Master1,Master2,Master3