Hi Jais,   What is the state of the Yarn application launched for processing the Hive query ? Is it : accepted, running, success, failed, killed ?   If it's in the state "accepted" then this is a Yarn configuration issue. For the other states, you will need to check the logs in order to understand what is happening : - of each tasks - of the HiveServer2 server used     ... View more

Thank you for the reply.   I know of the configuration that could be made at the workflow level. But our problem was with "solrctl" as stated. This utility is not optimized.   We have provided a FIX to Cloudera support on "solrctl" (with this fix, solrctl run with a very low memory profile). It was 'accepted' by the engeneering team but not sure when (or even if) it will be available in CDH. I have not tested the latest CDH version. So I don't know if it was integrated or not yet.   http://community.cloudera.com/t5/Cloudera-Search-Apache-SolrCloud/solrctl-issue-implentation-issue/m-p/49291#M732   By the way, I am "mathieu.d" if you were wondering (I changed my account).   ... View more

And if your sqoop command launch a map/reduce job then you also need to add the IP of all the data-nodes (where there is a Yarn NodeManager). ... View more

Ok, So you are experiencing GC issue.   First I would investigate the "client" for any unclosed connections or equivalent that could explain the "leak". Then only, I would investigate on the "server" side.   We have not run on that issue but we are not using the same CDH version and with an HBase configuration more powerful (more memory).   Good luck !       ... View more

HOST and PORT should target an active "HiveServer2" rôle. The default port is likely to be 10000. As for the host, well it depends on your cluster. ... View more

Not sure was to say. The graphic do not show any issue (on its own).   That is expected that the memory increase overtime and will decrease only when the Garbage Collector trigger itself (when needed - and as per the configuration).   There would be an issue, only if the memory increase leads to an OutOfMemory OR to much CPU consume by the GC. Are you experiencing that ?     ... View more

@pdvorak wrote: Beyond files, metastore tables and Hive SQL queries are also supported. Does that part of your answer suggest that MRIT supports Hive queries as data source for Solr indexing ? If yes how ? ... View more

For MLOPES they are running CDH 5.10.1 after a migration of CDH. With a fresh install of CDH 5.10.1 there is no issue. ... View more

For Impala there is a parameter that limit the amount of memory each deamon can use. Impala > Impala Daemon Default Group > Resource Management > mem_limit.   If you set 10 Go hear then each Impala daemon will use up to 10Go of memory.   This amount of memory is not part of the memory allocated to Yarn. So you should be careful.   ... View more

Try setting that value to 10 for example and retry. Do not modify the hardware specs. Just modify in Yarn the number of vCPU allocated.   You can to some extend "overcommit" the vCPU allocated to Yarn. For some testing cluster you should be fine.   Also, be sure you have allocated enough ram to Yarn.     ... View more

How many vCPU are allocated to yarn ? If the answer is 4 or less, then you don't have enough vCPU allocated to yarn for this to work. ... View more