devops tools used in Ambari

[#1]

Hi All,

With all the DevOps tools available (Docker, swram, kubernetes, Ansible, Puppet, Chef etc.) around configuration management, cluster management, container clustering etc.

I was wondering where does Amabri fits in, how is it developed? What about Ambari blue prints?

Thanks,

Avijeet

[#2 · Accepted solution]

@Avijeet Dash

At its core, Ambari is able to perform the same common ops that DevOps are used to with Chef and Puppet. Chef, Puppet, and Ansible are great for small clusters and limited operations, but as soon as you need to do perform complex operations like Kerberize, HA, Stack Upgrade, then those tools are not sufficient; this is where Ambari shines since it has deep knowledge about the cluster and Hadoop in order to perform things like:

• Rolling Restart (pick any component, like DataNodes, and pick your batch size, sleep time, and failures tolerated)
• High Availability for NameNode, Hive, Oozie, Resource Manager, etc (wizards to configure these and restart affected services)
• Config Management (this is where it truly brings value; if a config changes, Ambari can suggest other configs to change, and validate dependent configs)
• Kerberos Security: create and distribute keytabs, also handle use cases like adding hosts and more services
• Rolling and Express Upgrade: the orchestration and config changes here alone, plus handling a downgrade, would easily take weeks to develop and test. Ambari has all of the tools to visualize the orchestration, auto-retry, handle failures, skip optional steps, etc.
• Metrics: dashboards to visualize and analyze metrics, trends
• Alerts: notifications, SNMP, port checks, liveliness checks, and UI to enable/disable
• Maintenance Mode: put hosts and services in maintenance mode to suppress alerts and operations from running on certain hosts
• LDAP: active directory for enterprise clusters
• Service Auto-Start: automatically restart down services

and more.

Ambari was developed with scale in mind to minimize to support clusters with several thousand nodes. The agents heartbeat to the server with the status of commands and daemons; while the server sends commands down to the agents.

Blueprints is the mechanism to export an existing cluster and re-create it (or several others just like it) with a predefined topology, configs, etc. This is extremely useful in cloud environments.

I hope this answers your question.

[#3]

Thanks @Alejandro Fernandez

It will be great if you could share what technology/language it uses underline, if any. also custom ambari-services. How are those built.

[#4]

@Avijeet Dash, you may find this presentation useful, http://events.linuxfoundation.org/sites/events/files/slides/ambari_big_data_2017.pdf Custom Services can be added to the stack, as long as they support lifecycle commands like stop, start, restart, status, etc., and can also contain Custom Commands.

Technology-wise, Ambari Server is all Java and supports multiple DB types (Postgres, MySQL, Oracle, Azure). The Agents use Python for running, heartbeating, and executing the scripts for managing the Hadoop Services. The Agents send heartbeat messages to the Server every 4-10 seconds (configurable).

The web UI uses AngularJS and Node.js to responsiveness and auto-refresh, and relies on the RESTful web service for all data.