Member since
09-12-2016
39
Posts
5
Kudos Received
0
Solutions
01-09-2017
06:05 AM
@mqureshi Thank you for your answers. I want ask one more question. If I change just only on Ambari UI. So Is it equal with I used setrep command ? Or I need configure on Ambari UI before use setrep
... View more
01-09-2017
04:41 AM
I have two question about dfs.replication parameter: 1. I know default of replication block is 3. But when I configure dfs.replication=1, Do it affected to cluster performance. 2. I have a lot of data with configure dfs.replication=1, and now I change configure to dfs.replication= 3. So my data will auto replicate or I have to build my data again to replication running. I need to be sure because my data is very important. P/S: any best practice for dfs.replication configure.
... View more
Labels:
- Labels:
-
Apache Hadoop
12-21-2016
06:36 AM
@Rajkumar Singh I find this doc from hortonworks https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_yarn-resource-management/content/setting_application_limits.html and this from apache hadoop: I know if I change yarn.scheduler.capacity.maximum-am-resource-percent then Max Application Master Resources will change. But I want to know which configure it depend to calculate. For example if true I set yarn.scheduler.capacity.maximum-am-resource-percent = 1 (100%) then Max Application Master Resources balance with yarn.nodemanager.resource.memory-mb
... View more
12-21-2016
05:37 AM
1 Kudo
@Rajkumar Singh
So how can I define Max Application Master Resources. In default Configured Max Application Master Limit is set to 20%. If I increase figure, Max Application Master Resources increase. But how this Max Application Master Resources is calculated ?
... View more
12-21-2016
03:17 AM
1 Kudo
When I running yarn app, I often get some app in queue( state ACCEPTED). But you can see I still have free resource. So why app in queue ?
... View more
Labels:
- Labels:
-
Cloudera Manager
12-19-2016
05:46 AM
@Sunile Manjee Do you have any documents relate to this (cpu scheduling) ?
... View more
12-19-2016
05:30 AM
@Timothy Spann Yarn app not big so much, I run Apache Kylin and buid cubes. When running, Yarn app follow my memory setting. But for CPU, even when I set limit for CPU, it always exceed over my limit. I mean when I set CPU limit 80%, but when app running , it reach to 100% even 200%
... View more
12-19-2016
02:51 AM
1 Kudo
In my cluster, when I running an Yarn app, my CPU of each node get very high. I want to prevent this issue. I configure yarn.nodemanager.resource.percentage-physical-cpu-limit=80(%) But nodes still run out of CPU ( some time is 200% CPU cause I can not ssh to this) and also cause high CPU, hbase region server force to stop. I need limit CPU usage of nodes that mean when I run Yarn app, the CPU can not run out of resource. How can I do that. Thanks
... View more
Labels:
- Labels:
-
Apache YARN
11-29-2016
07:55 AM
Hi All, Below is my default queue( that use 100% YARN resource) So I wonder how Max Application Master Resource is calculated ?
... View more
Labels:
- Labels:
-
Apache YARN
10-31-2016
05:29 AM
@mqureshi thanks for your answer. So about the performance two nodes is the same one node if they have the same total hardware( and also the same if I spare to 5 VMs node) ? Can you suggest me small cluster design for 3 or 5 node. I want use this on production. I find the typical cluster referent from document Masters -- HDFS NameNode, YARN ResourceManager, and HBase Master.
Slaves -- HDFS DataNodes, YARN NodeManagers, and HBase RegionServers
... View more
- « Previous
-
- 1
- 2
- Next »