Community Articles
Find and share helpful community-sourced technical articles
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Expert Contributor

In Hortworks HCP they use an ElasticSearch Mpack for version 5.x of Elasticsearch and Kibana:

I have taken this and expanded it to include version 6.3.2 of ElasticSearch, Logstash, Kibana, FileBeat, and MetricBeat.


My cluster is 6 nodes.

ElasticSearch is installed on Nodes 4,5,6(4 Master & 5,6 Data Nodes).
Logstash is Installed on Node 3.
FileBeat & MetricBeat are installed on all 6 nodes.
Kibana is installed on Node 4.
The rest of the cluster is configured normally for a minimal Install.


Orginal HCP Mpack


My MPack 6.3.0 - Version Change + Logstash + Beats


My Mpack 6.3.2 - Version Change + Logstash (multi-node & jvm settings config) + sudo non root capabilities


Installation Steps

1. Deliver Management Pack tar.gz to local filesystem on Ambari-Server

upload to hdfs view Files View

download to ambari-server node (/home/root)

2. Install Management Pack

sudo ambari-server install-mpack --mpack=/home/root/elasticsearch_mpack- --verbose

Uninstall is: sudo ambari-server uninstall-mpack --mpack-name=elasticsearch-ambari.mpack

3. Restart Ambari Server

sudo ambari-server restart

4. Use Ambari Add Service to Install ELK Stack Components

The following settings will be required during the Install Wizard




ElasticSearch Zen Discovery Hosts:
example: [,, ]


Post installation Ambari handles the configuration of all components including logstash (input, output, and filters) and FileBeat and MetricBeat configuration files.

  • ElasticSearch configuration should work out of the box without any changes other than Zen Discovery Hosts.
  • Logstash filters are setup for beats input, file FileBeat filter, and elasticsearch output.
  • FileBeat is setup to use Logstash.
  • MetricBeat is setup to send metrics directly to Elasticsearch.


Learning this Elasticsearch MPack is a good example of how to create your own custom stack using a Management Pack to define services not normally found in an Ambari Cluster. Ambari Administrators looking to understand how to create their own Management Pack should take some time to diff the Mpacks attached below. Creating custom services controlled via Ambari is pretty easy if you mimic the folder structure, make the necessary xml file changes, and adjust the python package scripts accordingly.

0 Kudos

@ Steve Matison

Thank you for posting the ES MPACK. Would you be able to share how to build our own custom MPACK. Maybe a follow on blog.



Expert Contributor

@Amit Nandi sure I will work on an article for how to make an mpack.

Expert Contributor

Today I am working on an ELK Mpack for HDP 3.0+ and HDF 3.0+. To pick this back up I have started on another documentation journey to record my progress. Starting with this post and I was able to get a base cluster up with the ELK above. I will be creating another article for getting this to work on the new HDP stacks as well as changes for most recent version of ELK stack.

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 06:35 AM
Updated by:
Top Kudoed Authors