Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar

How within few minutes you can setup Rack Awareness through Ambari?

Series1: Introduction to Rack Awareness can be viewed here:

https://community.hortonworks.com/content/kbentry/43057/rack-awareness-1.html

How does my cluster look?

Image below shows the hosts and the rack:

5449-r1.jpg

You will see that the rack mentioned for each of the hosts in the test cluster is “/default-rack”. Which means Ambari (And HDFS and YARN) thinks of this cluster inside a single rack i.e. default-rack. In other words, it is not rack aware.

Now let us examine the configurations that will change after we make the cluster rack-aware.

Steps to verify Rack Awareness:

  • 1.Login to any node of your cluster. I am choosing node2.
ssh root@node2
  • 2.List the configuration files for rack awareness and view the current mapping of the system:
ls –lrt /etc/hadoop/conf/topology*

cat /etc/hadoop/conf/toplogy_mappings.data

5450-r2.jpg

Qs: Why is node 2 not listed here?

Because node 2 does not contain a datanode.

5451-r3.jpg

  • 3.As super user run the fsck and dfsadmin commands
su – hdfs

hdfs fsck –racks

hdfs dfsadmin –report

5452-r4.jpg

For showing the relevant entries I grepped “hdfs dfsadmin –report” command. You will see that there is no rack information attached to the current cluster

5453-r5.jpg

Steps to setup Rack Awareness through Ambari:

  • 1.Log in to Ambari UI
  • 2.Click on Hosts tab

5454-r6.jpg3.Click on Individual hosts and then click on Host actions:

5455-r7.jpg

4.Click on Set Rack in the host actions and set the rack name (I choose two racks: rack1 and rack2). Then click ok.

5456-r8.jpg

5.Hit Back and go back to the Hosts page.

5457-r9.jpg

Similarly set rack names for the other nodes in your cluster.

So far you do not need to restart any components.

6.I have set up the following rack names for the different nodes in my cluster:

5458-r10.jpg

7.Now go back to your dashboard and you will see that HDFS and MapReduce2 services needs to be restarted

5459-r11.jpg

8.Restart those two services.

5460-r12.jpg

5461-r13.jpg

5462-r14.jpg

5463-r15.jpg

Wait for them to finish and your cluster is now Rack Aware

Steps to verify Rack Awareness:

1.On the same terminal view the current topology mapping of the system:

cat /etc/hadoop/conf/toplogy_mappings.data

5464-r16.jpg

As you can see the racks are mapped as we intended through Ambari Admin console.

2.As super user run the fsck and dfsadmin commands

su – hdfs

hdfs fsck –racks

hdfs dfsadmin –report

5465-r17.jpg

The report shows that Rack Awareness is in effect.

Logout from superuser aka hdfs in this case.

5466-r18.jpg

Congratulations your cluster is now Rack Aware!!!

9,077 Views
Comments
avatar
New Contributor

After making the changes as specified for rack awareness, the resource manager (resourcemanager:8088/cluster) still shows the rack as /default-rack. Is this expected? Why doesn't the resource manager show the correct rack information?

avatar
Super Guru

@rbiswas

What about adding new host and assign it at the same time to a specific rack? I'd like to avoid the host (data node) and then have to set the rack and restart again. Is there a way via Ambari UI?

Version history
Last update:
‎08-17-2019 11:43 AM
Updated by:
Contributors