Alternatively you could search around "yarn queue" and ressource allocation.   This will not "restrict" the number of mappers or reducers but this will control how many can run concurrently by giving access to only a subset of the available resources.     ... View more

@Harsh J, Thanks for quick reply. I thought the ouptut of fsck command includes replica multiplier and gives final total block count. Thanks for the clarification. I checked Datanodes page on namenode WebUI and block count for each datanode is more than threshold value. Thanks, Priya ... View more

If I have 50 users. 10 users are updating and inserting the same base table  simultaneusly and 40 users are just querying simultaneously, will there be any locking? How is the concurrency working?   I do not want to turn on the transactional or ACID features. Please let me know.   Thanks, Raja ... View more

there are couple of places that needsd tuining in the query level    1 . stats for the table is must for good performance  2.  when user is joining two tables make sure there are using the large table in the last and the first table is smaller  3. you can also use HINTS to imporve query performance. 4. hive table's file format is big a factor  5. choosing when to use paritioning vs bucketing.    6.allocate good memory to hiveserver2 and metastore  7.heapsize  8 .load balancer on the host    https://www.cloudera.com/documentation/enterprise/5-9-x/topics/admin_cm_ha_hosts.html#concept_qkr_bfd_pr ... View more

Thanks! Yes I was able to set this using the CM Hive->configuration->Scope Hive (Service wide). Then restart the Hive service.   Hive Service Advanced Configuration Snippet (Safety Valve) for sentry-site.xml Hive (Service-Wide)    Name: sentry.hive.testing.mode    Value: true   Description     Final ... View more

As an alternative, you could enable LDAP for Impala and then connect to the slaves directly thus bypassing Kerberos and the load balancer. ... View more

@ni4ni   Yes that is not the right place, according to the link that i've given above this configuration change should go to core-site.xml, so search for    "Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml" and add/modify as needed and restart the namenode ... View more

@saranvisa Thanks for answers! 1. Yes, we have Grant role TO user 2. We try it in HUE 5. Yes, Sentry service is enabled. I added Grant all on server, and with this grant all works fine... ... View more

The amount of memory to assign to the JVM is relative to the number of documents in solr core nav_elements as per the documentation. See role log to get this number from your instance. The JVM sizing formula is number of nav_elements * 200, which gives you a rough estimate of what is required for normal operation.  ... View more

If you want to use spark2-shell and spark2-submit, you don't have to set those ENV variables. I set it because I wanted to point the current spark-shell/submit to spark2.   This should be done in all the nodes that you want to use the shell and/or the submit.   I forgot to add the changes that I made for spark-sumbit. In these files:   /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/bin/spark-submit /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/spark/bin/spark-submit Add this ENV var: SPARK_HOME=/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2 ... View more