Alternatively you could search around "yarn queue" and ressource allocation.   This will not "restrict" the number of mappers or reducers but this will control how many can run concurrently by giving access to only a subset of the available resources.     ... View more

I think what you search is a configuration located inside the "core-site.xml" file (in HDFS configuration). search for "proxyuser" on the documentation of Cloudera.   regards, Mathieu ... View more

there are couple of places that needsd tuining in the query level    1 . stats for the table is must for good performance  2.  when user is joining two tables make sure there are using the large table in the last and the first table is smaller  3. you can also use HINTS to imporve query performance. 4. hive table's file format is big a factor  5. choosing when to use paritioning vs bucketing.    6.allocate good memory to hiveserver2 and metastore  7.heapsize  8 .load balancer on the host    https://www.cloudera.com/documentation/enterprise/5-9-x/topics/admin_cm_ha_hosts.html#concept_qkr_bfd_pr ... View more

Do you need the --override?   I reran my Tutorial 1 and it didnt append new records....I thought it would...why do you think it allowed it? ... View more

Finding logs manually in machine sound very brute force; I was thinking more of an API or CLI option to find logs   Anyway the main issue we're trying to solve is access to logs to all developers in prod environment. Our node managers are behind the bars and not accessible ( any port or web ) to develoeprs and it's unlikely to happen. So we're trying to find a way to proxy the logs.   I discovered that there is a jobhistory proxy to look at completed jobs / yarn apps but I coudln't get it working for running app. Is there any trick / way to access running app's logs like above ?   http://resourcemanager.xyz.com:19888/jobhistory/logs//dataNode.com:8041/container_id_000001/container_id_000001/root     ... View more

Thank you for the detailed answer ... View more