About DKL

DKL · ‎08-20-2014

After playing around I have found that in order to actually get Cloudera Manager to recognize a change in RAM on the system, you have to restart the cloudera-scm-agent on whichever host got the change in RAM. Once the restart has completed Cloudera Manager will automatically update with an updated value of physical memory.

DKL · ‎08-19-2014

That's really helpful information! Thank you!! So, if I am getting this right, we would use an external database for production because it gives us more control of database options and configurations during install which become imporant especailly for larger clusters? Thank you again

DKL · ‎08-19-2014

We have a cluster with 36 nodes and have initially installed Cloudera Manager with an embedded database. The only services currently installed are HDFS and YARN (MR2). Now, I would like to properly install Hive and Impala on this cluster. The Question: What is the difference between an embedded database and an external database? To me, it seems like embedded postgreSQL is just a script that creates all of the databases for you, where as with an external you do it on your own. If this is the only difference, it must be okay for me to use the same postgresql installation to create an "external" database for Hive, right? An Issue I ran into previously: One time, I followed the documentation to install Hive exactly, which included doing a yum install of postgres. I didn't inspect enough and had just performed the installtion. Little did I know, I had actually just installed a second version of postgreSQL. I had 9.2 from the yum and 8.4 from Cloudera Manager. Everything was working for a while, but then some unrelated error occurred and caused the Cloudera Manager database to stop. When trying to start the database again, Cloudera Manager attempted to connect to the 8.4 db using 9.2 commands. Woops... lesson learned here. My Goal: Ultimately, I am trying to use postgreSQL for both Cloudera Manager and Hive/Impala. From my previous lesson, I also learned that it would probably be best to keep only one installation of postgreSQL too. What would be the proper way to install? Thank you!!!

DKL · ‎08-05-2014

Firstly, thank you for your response and I apologize for the lack of description. I'll attempt to be more descriptive here: Where am I seeing this: Go to "Hosts" tab, click on any of the datanodes (4 total), in the top left of the datanode status page there is a "Details" section. In there, there is a "Host Agent" row which displays "Physical Memory" as a bar that fills up. (Shown Below) What it states: It states that it has 8GB total of memory available. I feel like it should be more now though given the increase in memory we gave to the datanodes. Why I feel this is wrong: We had initially only had 8GB on each node, but then we bumped up the memory on each node. Now when doing "cat /proc/meminfo" on one of the datanodes, it shows that at least 25GB is free. So, to me, I feel like Cloudera Manager has to somehow reevaluate how much memory is available on the datanodes. Does this help?

DKL · ‎07-29-2014

We installed Cloudera Manager with only 8GB of memory on the datanodes. Later, the memory was bumped up, however Cloudera Manager still shows that each datanode can only utilize 8GB of physical memory (shown in the hosts tab and it will also complain of overcommit if more than 8GB is allocated total). Is there a way to update Cloudera Manager with the newly added memory or is there a configuration to adjust this? Thank you!

Online	Offline
Last Visited	‎09-26-2014 02:44 PM

Member Since	‎07-29-2014 09:23 AM
Last Visited	‎09-26-2014 02:44 PM
Posts	9
Kudos received	3

Cloudera Community

Re: physical memory aware to cloudera manager

Re: physical memory aware to cloudera manager

Re: Embedded vs External database?

Embedded vs External database?

Re: physical memory aware to cloudera manager

physical memory aware to cloudera manager