Hi, Your CDH version is 5.1.2. What is the Cloudera Manager version? Note that these can be different numbers. Cloudera Manager allows you to add either Spark on YARN and Standalone Spark. These aren't different versions of the Spark binaries, they're different ways to deploy spark. In Cloudera Manager, these two are different service types. Which service type did you install? If you aren't sure which you are using, look at the Instances tab for your spark service. If you have any Master or Worker roles, then you're using Standalone Spark. Are you using Spark on YARN or Standalone Spark? Thanks, Darren ... View more

Hi, Are you using spark on yarn or standalone spark? Note that standalone spark doesn't work with Kerberos. Which version of CM are you using? Which version of CDH? Thanks, Darren ... View more

What do the role logs say? It sounds like this is a brand new Sqoop 2 (not an upgrade). Are you sure you gave it an empty database? Check /var/lib/sqoop2 (or whatever you've configured for "Sqoop 2 Server Metastore Directory") for any leftover data that may be causing a problem. Thanks, Darren ... View more

Hi. Agreed with Gautam. If you want to do this via the API, then you can configure process_auto_restart in the appropriate role config group. If you want to restart particular roles instead of the whole service, use the following API endpoint: http://cloudera.github.io/cm_api/apidocs/v9/path__clusters_-clusterName-_services_-serviceName-_roleCommands_restart.html Thanks, Darren ... View more

Hi Mugi, The docs I linked offer some guidance for things like max connections, which can help determine the sizing of your DB hardware. Usually the DB load from CM and monitoring is not extreme, but for 20+ node clusters we recommend splitting all monitoring roles and databases to be on a separate host than CM server to spread the load. How many can be shared on a host depends on cluster size. I'm frankly not an expert in this area, but others may be able to chime in. You can also of course get official support for your cluster and get reliable access to this kind of expertise (</shameless plug>). Thanks, Darren ... View more

There's an option to customize where the Cloudera Manager agent binaries come from. Look carefully at each page of the wizard. You may need to scroll down on one of the pages to see it. You can also do a Path B install, ie install the agents yourself instead of having CM do it. ... View more