https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98694#M61794 <P>What’s the approximate load on DB if we collocate ambari, oozie, hive, ranger admin &amp; audit etc on the same DB cluster? </P><P>- Cluster Size ~ &lt;100 nodes, 100-500 nodes, 500-1000 nodes and 1000+</P><P>- Number of Users for Hive and Oozie ~100+</P> Fri, 11 Dec 2015 05:55:53 GMT pardeep_kumar 2015-12-11T05:55:53Z https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98694#M61794 <P>What’s the approximate load on DB if we collocate ambari, oozie, hive, ranger admin &amp; audit etc on the same DB cluster? </P><P>- Cluster Size ~ &lt;100 nodes, 100-500 nodes, 500-1000 nodes and 1000+</P><P>- Number of Users for Hive and Oozie ~100+</P> Fri, 11 Dec 2015 05:55:53 GMT https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98694#M61794 pardeep_kumar 2015-12-11T05:55:53Z https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98695#M61795 <P>Is the host only going to contain DBs, or will it also contain Ambari Server, HiveServer, etc?</P><P>Spreading it out is wise, but if you're really constrained, the Ambari DB is usually no more than 100 MB. There's an article on how to optimize the Ambari DB for large clusters (200+ nodes) <A href="http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.1.0/bk_ambari_reference_guide/content/ch_tuning_ambari_performance.html">http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.1.0/bk_ambari_reference_guide/content/ch_tuning_ambari_performance.html</A></P> Fri, 11 Dec 2015 05:59:40 GMT https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98695#M61795 afernandez 2015-12-11T05:59:40Z https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98696#M61796 <P><A rel="user" href="https://community.cloudera.com/users/139/pardeepkumar.html" nodeid="139">@Pardeep</A> It will be very difficult to forecast the load. Assumption: All connections are active them load can be low to medium </P> Fri, 11 Dec 2015 06:00:26 GMT https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98696#M61796 nsabharwal 2015-12-11T06:00:26Z https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98697#M61797 <P>The base DB's are usually small in size.The Ambari Server, HiveServer components should just be DB schemas a golden rules is always to have failover configured and tested and documented.</P><P>Don't forget non of these DB's is OLTP ,latency shouldn't be an issue so a cheap rackable 24 to 36 GB RAM and 8 CPU should be fine </P> Fri, 11 Dec 2015 06:07:35 GMT https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98697#M61797 Shelton 2015-12-11T06:07:35Z https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98698#M61798 <P>I would be interested in planning for Hardware i.e. CPU and RAM for the database server(s). Disk requirement won't be high and not much concerning.</P> Fri, 11 Dec 2015 06:20:31 GMT https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98698#M61798 pardeep_kumar 2015-12-11T06:20:31Z https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98699#M61799 <P><A rel="user" href="https://community.cloudera.com/users/326/afernandez.html" nodeid="326">@Alejandro Fernandez</A>There is no guidance on Sizing database server i.e. CPUs, RAM required for Database Server for small to medium and large clusters.</P> Fri, 11 Dec 2015 06:22:28 GMT https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98699#M61799 pardeep_kumar 2015-12-11T06:22:28Z https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98700#M61800 <P>We had some performance issue with a low profile config (4 vCores, 8 GB RAM), expecially with Oozie. Right now we reccomend at least 4 vCores and 24 GB or RAM). If you're planning a IaaS deployment on Azure using as metadata repository SQL Azure start directly with a S2/S3 instance: if you use Oozie it's the minimum requirement.</P><P>Pay also attention to Ranger: it's OK to Ranger admin and users but for audits you need to look carefully at DB sizes: it can easly grow up faster. Use a script to truncate the table or use a different instance.</P> Sat, 12 Dec 2015 03:51:57 GMT https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98700#M61800 dorio 2015-12-12T03:51:57Z https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98701#M61801 <P>Just configure the Nagios and Ganglia those 2 monitoring tools should give you some good metrics</P> Sat, 12 Dec 2015 16:39:01 GMT https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98701#M61801 Shelton 2015-12-12T16:39:01Z https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98702#M61802 <P><A rel="user" href="https://community.cloudera.com/users/1085/dorio.html" nodeid="1085">@Andrea D'Orio</A> Re: Ranger DB</P><P>The recommended approach is to go with Solr instead DB "for all the new deployments" </P> Sat, 12 Dec 2015 20:58:40 GMT https://community.cloudera.com/t5/Support-Questions/Database-Sizing-and-recommendation-for-Ambari-and-HDP/m-p/98702#M61802 nsabharwal 2015-12-12T20:58:40Z