Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)

This article is a follow-up on my original article about the visualization of a cluster and its services/components ( https://community.hortonworks.com/articles/2010/visualizing-hdp-cluster-service-allocation.html). In the first part I am particularly focusing on a new feature that enables users to build and plan a cluster by using a drag-n-drop Web UI.


Build a Cluster


Until now visualizing a cluster and its service allocation either meant exporting the information from Ambari or writing a JSON file that outlines the details of the nodes. Planning and deploying a cluster should be easier, right?


I'll introduce, Build a Cluster


702-byoc-overview.png


This simple Web UI is based on different drag-n-drop functionalities and allows the creation of a new cluster by simply dragging Hadoop components from the elements list to the indivdual nodes.


Lets go over the different features....


User Interface


The UI is divided into two sections Elements & Settings and Cluster:


  • Elements & Settings (left): Contain the available services and components of the environment (remember these can be edited by simply importing a different env.) as well as cluster settings (HDP version, cluster name, security enabled yes/no). Additionaly this section provides some action buttons to finalize the cluster and add nodes
  • Cluster (right): This is the current cluster with all its components. Elements can be dragged from the elements list and dropped in the individual nodes. Nodes can be edited or removed.


Nodes


Note: The data structure of Nodes has changed in this version, one node does not have to represent a single physical machine anymore, a node in this app can now represent many physical machines that all share the same components.


703-byoc-node-elements.png


Adding Nodes


The number of nodes is curently limited to 1000.


New nodes can be added by pressing the "+ Node"-button in the elements section.


Editing the hostname and cardinality


Simply click on the hostname or cardinality. (Note: ever


704-byoc-node-hostname-edit.png


Hostname Syntax


Hostnames allow some special syntax, which automatically generates multiple hostnames (only if cardinality is set > 1)


#{x} => Number with trailing zeros


{x} => Number without trailing zeros


X => defines the start of the counter


Examples:


1) datanode#{0}.example.com (Cardinality = 2)


  • datanode1.example.com
  • datanode2.example.com


2) datanode#{0}.example.com (Cardinality = 30)


  • datanode01.example.com
  • datanode02.example.com
  • ...


3) datanode{100}.example.com (Cardinality = 20)


  • datanode100.example.com
  • datanode101.example.com
  • ...


4) datanode.example.com (Cardinality = 2)


  • datanode.example.com1
  • datanode.example.com2


Adding components to a Node


  1. Select a service in the Elements list, this will bring up the list of components of this service
  2. Select & Drag a component to any of the nodes


707-byoc-add-comp-to-node.jpg


Removing a component from a Node


Drag the component from the node and drop it outside the node or over the "Trash" area, inside the elements section


Finalize a Cluster


When your cluster is finished, press the "Finalize"-button inside the elements section, this will convert the built cluster into the same data format as any exported or JSON-specified cluster. Additionaly this imports or basically transfers the new cluster to the main page "Cluster". Finalizing a cluster also regenerates the Ambari Blueprint (read more in the next section)


Note: You can press the button multiple times while you're developing a new cluster This might be helpful, e.g. if you want to see different views (service, component, list) on the Cluster-page during the development.


712-byoc-finalize-low.png


Generating Ambari Blueprints


In this section I am focusing on another new feature that will more than simplfy the creation of Ambari Blueprints.


709-bp-result-low.png


The blueprint section contains the actual blueprint (left) as well as the cluster creation template or hostgroup mapping (right). Hitting the "Copy"-button in the upper-right corner will copy the individual content to the clipboard (might not work with all browsers)


Cluster-Configuration


No blueprint is complete without any configuration! The Cluster-Configuration page provides the necessary functionalities to add general or host-group specific configurations to the blueprint or the cluster creation template (hostgroup mapping).


711-bp-config-overview-low.png


I have seen plenty of blueprints that had typos within the configuration section, e.g. instead of dfs.blocksize the blueprint included a dfs.blcksize configuration. This is why the typeahead feature for the config location and name was added. Simply start typing and the app will come up with some suggestions.


710-bp-typeahead.png


HDFS HA & Yarn HA configurations (automation)


A nice little gimmick that has been added to this application is the automatic config generation for HDFS HA and Yarn HA clusters. Whenever the app recognizes a specified set of service components (e.g. 2 Namenodes, 3 Journalnodes, etc.) in the cluster, it will automatically generate the necessary configuration for HDFS or Yarn High Availability.


Project & Setup: https://github.com/mr-jstraub/ambari-node-view


I hope you enjoy these new features and find them useful.


Looking forward to your feedback and feature requests

2,455 Views
Comments

@Jonas Straub, Your app is excellent to visually build a cluster. but i am not able to change the host names in the app. If i edit the host name it again goes back to the old host name like "master01.example.com". Is it possible to edit the names.

Thanks

atun

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 01:42 PM
Updated by:
 
Contributors
Top Kudoed Authors