I agree. HDP 2.6 is very solid. We have thoroughly tested against 200k queries/hr (ootb is about 40k/hr, need to go to local metastore, increase appmaster's, etc. to increase). HDP 2.5.3 was solid too, though technically LLAP was tech preview in that release. ... View more

@Benjamin Leonhardi , on slide 24 you notate that a small stripe size indicates a memory problem during load. Do you know what memory problem that would be? I have ~ 3500 records on the stripe and was just wondering where I should look. Thanks! ... View more

@Constantin Stanca yes, in-memory cache per daemon = 12288 (Mb) ... View more

yes - it is in advanced hive-interactive-site. Also notice, there is no set-recommended button (blue 3/4 circle arrow). Thanks again Constantin. ... View more

I should also add everything looks correct, starts correctly and runs well. It just isn't using the io cache. I do have millions of these messages in yarn timeline server log, but from looking at the orc writer code it doesn't look like orc writers uses timeline server: 2017-02-21 00:20:36,581 WARN timeline.EntityGroupFSTimelineStore (LogInfo.java:doParse(207)) - Error putting entity: dag_1487651061826_0016_921 (TEZ_DAG_ID): 6 Other than that, everything is clean. Thanks again, I appreciate it. ... View more

hivellapiomemorymode.png Yeah, I thought so too. It was literally blank (see screenshot of first ambari version post enabling interactive query). Also odd was the hive.llap.daemon.vcpus.per.instance, it was a variable rather than the number. This generated a WARN at start ("WARN conf.HiveConf: HiveConf hive.llap.daemon.vcpus.per.instance expects INT type value"). So I put 32 in for the vcpu's and I put cache for hive.llap.io.memory.mode. They saved correctly to hive-site.xml. The third anamoly is I get is a message that says "WARN conf.HiveConf: HiveConf of name hive.llap.daemon.allow.permanent.fns does not exist". This comes from ambari and I can remove from hive-site.xml, but ambari will put it back. Everything else was as I would expect. hive.llap.io.enabled is true. According to Grafana, I have a 12Gb cache, but there isn't anything in it. Thanks for your help, I appreciate it.. ... View more

yes, typo in post. it is hive.llap.io.memory.mode=cache Thanks! ... View more

@Prasanth Jayachandran @Serge Dyshlevyi ... View more

Update for those out there thinking of answer, (and a cheat sheet for those browsing of good parameters to set - I have tested each individually and have response time down to 630ms and throughput to 40 queries/sec for a 1Tb dataset, but need to get the iocache going to get to the next level and improve throughput (q/sec)): The io cache is created (12Gb), but is never used. I am using OOTB LLAP settings except for below changes, all of which were positive in driving down response time and increasing throughput, except a few were neutral: -increased mem for yarn, llap heap, llap concurrency -disabled CBO (it's adding time to query during creating and submitting plan portion of query) -set hive.llap.iomemory.mode=cache (OOTB this setting was null) -increased /proc/sys/net/core/somaxconn to 4096 -increase metastore heap -increased yarn threads (yarn.resourcemanager.amlauncher.thread-count) -increased Yarn Resource Manager heartbeat interval (tez.am-rm.heartbeat.interval-ms.max) -set hive.llap.daemon.vcpus.per.instance to 32, previously it is unrecognized variable (ambari bug) and gave message in ambari startup of "WARN conf.HiveConf: HiveConf hive.llap.daemon.vcpus.per.instance expects INT type value" -added additional HS2 instance. -added the 4 hive-conf options (hive.execution.mode, hive.llap.execution.mode, hive.lllap.io.enabled, hive.llap.daemon.service.host.) as additional properties in HS2 custom site. -increased HDFS heap Version of HDP is 2.5.3.0-37 Version of Hive is 2.1.0.2.5.3.0-37 ... View more