Hi Zealot,   How much ram does your machine have? How many roles did you assign to that host? How many hosts? If you had 64GB or more of ram, you would not be hitting this problem. Probably 32GB would also be ok, at least in terms of getting rid of these warnings.   It sounds like your machine doesn't really have enough memory to handle the roles assigned to it, so when CM automatically tuned the memory to try and split it between the roles, they ended up being below the minimum recommendations. You can configure your various roles to increase their memory (clicking on each of those messages will lead to the relevant config page), but this will just mean that your roles are trying to use more memory than your host physically can provide, so you'll see performance problems or crashes.   You should probably try to use a larger machine or reduce the number of services in your cluster so that everything has enough memory.   Thanks, Darren ... View more

Are you looking at this guide? http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managing-Clusters/cmmc_HDFS_hi_avail.html This guide talks about configuring HDFS for High Availability. It does not talk about installation. Did you already add a cluster with services like HDFS and ZooKeeper configured? I assumed from your post that you had an existing cluster. You enable High Availability after you've created a cluster. If you don't have such a cluster, then add one and configure the desired services, then follow the High Availability guide. ... View more

Hi Nishan, Here's how you can find documentation: 1) go to http://www.cloudera.com/content/support/en/documentation.html 2) Click on your version of Cloudera Manager (4 or 5) 3) From the dropdown, select the correct minor version of Cloudera Manager, or the closest number to it. 4) Click on the Installation Guide 5) Look for your topic in the left, or use the search in the upper left. In your case, you can search for "High Availability" and you'll find the right document for turning on HDFS High Availability. Thanks, Darren ... View more

Hi Graham,   This is a service-level property of HDFS. The rest endpoint to find it looks like this: http://<cm_host>:7180/api/v5/clusters/<cluster_name>/services/<hdfs_service_hame>/config?view=full   To modify it using the python api, get a handle to the HDFS service and call hdfs.update_config({"core_site_safety_valve": core_site_safety_valve_value }).   See more on using the python API here: http://cloudera.github.io/cm_api/docs/python-client/   Thanks, Darren ... View more

Hi Nishan,   During the install wizard, there's a prompt for you to specify where you'd like the CM agent binaries to come from. It's on the same page that prompts you to select CDH version. By default, it will pick up the corresponding build from our public repos, but you can override that.   Thanks, Darren ... View more

Hi Nishan, It seems that you've accidentally deleted the log4j file and need to add it back. Here's what that /etc/cloudera-scm-server/log4j.properties should have as contents: # Copyright (c) 2012 Cloudera, Inc. All rights reserved. # # !!!!! IMPORTANT !!!!! # The Cloudera Manager server finds its log file by querying log4j. It # assumes that the first file appender in this file is the server log. # See LogUtil.getServerLogFile() for more details. # # Define some default values that can be overridden by system properties cmf.root.logger=INFO,CONSOLE cmf.log.dir=. cmf.log.file=cmf-server.log # Define the root logger to the system property "cmf.root.logger". log4j.rootLogger=${cmf.root.logger} # Logging Threshold log4j.threshhold=ALL # Disable most JDBC tracing by default. log4j.logger.org.jdbcdslog=FATAL # Disable overly loud Avro IPC logging log4j.logger.org.apache.avro.ipc.NettyTransceiver=FATAL # Disable overly loud Flume config validation logging log4j.logger.org.apache.flume.conf.FlumeConfiguration=ERROR log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender log4j.appender.CONSOLE.target=System.err log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} %5p [%t:%c{2}@%L] %m%n log4j.appender.LOGFILE=org.apache.log4j.RollingFileAppender log4j.appender.LOGFILE.MaxFileSize=10MB log4j.appender.LOGFILE.MaxBackupIndex=10 log4j.appender.LOGFILE.File=${cmf.log.dir}/${cmf.log.file} log4j.appender.LOGFILE.layout=org.apache.log4j.PatternLayout log4j.appender.LOGFILE.layout.ConversionPattern=%d{ISO8601} %5p [%t:%c{2}@%L] %m%n   Thanks, Darren ... View more