Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to mesaure Thrift2 performance and how to improve it

How to mesaure Thrift2 performance and how to improve it

New Contributor

Hi everybody,

We are experiencing some problems with some windows clients accesing HBase through thrift2, in particular timeouts and errors. We would like to know: how can we monitor thrift2 performance? I have been searching for it but there is not so much documentation about it.

Apart from it, are there any configurable parameters to start the service for more performance? We start using "/usr/hdp/2.3.2.0-2950/hbase/bin/hbase-daemon.sh start thrift2".

We are using version package "hbase_2_3_2_0_2950-thrift2-1.1.2.2.3.2.0-2950.el6.noarch" on "CentOS Linux release 7.2.1511 (Core)".

Regards,

Silvio

8 REPLIES 8

Re: How to mesaure Thrift2 performance and how to improve it

Super Collaborator

You can try specifying read timeout (default is 60000):

options.addOption("t", READ_TIMEOUT_OPTION, true, "Amount of time in milliseconds before a server thread will timeout " + "waiting for client to send data on a connected socket. Currently, " +

Re: How to mesaure Thrift2 performance and how to improve it

New Contributor

Thanks for your answer. Currently we have two thrift2 servers on two machines, both machines are identical (hardware speaking), same VLAN and different IP. In one of them, queries to HBase through thrift2 work like a charm but, in the other one, they work veeery slow, with even timeouts and errors. In this last one, working slow, there are many queries, more than in the other. That's why I would like to know how to measure performance and what parameters shoud I monitor.

Regards,

Silvio

Re: How to mesaure Thrift2 performance and how to improve it

Super Collaborator

You can specify the number of worker threads through w option:

options.addOption("w", "workers", true, "How many worker threads to use.");

Re: How to mesaure Thrift2 performance and how to improve it

New Contributor

I'm sorry because I'm quite newbie :-( How can I specify these parameters you are writing about? I start thrift2 process from command line with "/usr/hdp/2.3.2.0-2950/hbase/bin/hbase-daemon.sh start thrift2"

Highlighted

Re: How to mesaure Thrift2 performance and how to improve it

Super Collaborator

Here is a sample command:

/usr/hdp/2.3.6.0-3796/hbase/bin/hbase-daemon.sh start thrift2 w 23

Re: How to mesaure Thrift2 performance and how to improve it

New Contributor

Ok, I found different options to pass as parameters to daemon. Default "t" is 60000 (60 seconds), do you know how many "w" workers are the default?

Re: How to mesaure Thrift2 performance and how to improve it

New Contributor

Hi there,

Ted, could you explain please the meaning of the "w" parameter, i guess you've check the thrift server source code:

  private static Options getOptions() {
    Options options = new Options();
    options.addOption("b", "bind", true,
        "Address to bind the Thrift server to. [default: 0.0.0.0]");
    options.addOption("p", "port", true, "Port to bind to [default: " + DEFAULT_LISTEN_PORT + "]");
    options.addOption("f", "framed", false, "Use framed transport");
    options.addOption("c", "compact", false, "Use the compact protocol");
    options.addOption("w", "workers", true, "How many worker threads to use.");
    options.addOption("q", "callQueueSize", true,
      "Max size of request queue (unbounded by default)");
    options.addOption("h", "help", false, "Print help information");
    options.addOption(null, "infoport", true, "Port for web UI");
    options.addOption("t", READ_TIMEOUT_OPTION, true,
      "Amount of time in milliseconds before a server thread will timeout " +
      "waiting for client to send data on a connected socket. Currently, " +
      "only applies to TBoundedThreadPoolServer");
    OptionGroup servers = new OptionGroup();
    servers.addOption(
        new Option("nonblocking", false, "Use the TNonblockingServer. This implies the framed transport."));
    servers.addOption(new Option("hsha", false, "Use the THsHaServer. This implies the framed transport."));
    servers.addOption(new Option("threadpool", false, "Use the TThreadPoolServer. This is the default."));
    options.addOptionGroup(servers);
    return options;
}

As you can see there is the "workers" parameter... but... if you see the TThreadPoolServer source code file, you could see that this parameter only modifies the number of workers to use upper bound:

    if (workerThreads > 0) {
	serverArgs.maxWorkerThreads(workerThreads);
    }

So, i guess this parameter is only used to limit the threading capacity of the server. Is this right?

And regarding the timeout parameter, as you could see on the first code snipet, it's only applicable to the TBoundedThreadPoolServer, which is no the deafult server.

Thanks in advance!

Re: How to mesaure Thrift2 performance and how to improve it

New Contributor

Anyone? How do you usually proceed with performance measurement? What tools do you usually use?

Don't have an account?
Coming from Hortonworks? Activate your account here