- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Deploying hadoop cluster
- Labels:
-
Apache Hadoop
Created ‎02-01-2016 09:32 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I want to set up hadoop cluster where i want master node and few slave node to be in some country and remaining slave nodes sitting in different country. Is it possible to deploy this? Is there any article or tutorial which might help me to get the solution.
Created ‎02-01-2016 10:04 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @Kabirdas B K yes that's possible, you can just add remote nodes like local nodes, and yes you can set rack awareness, but it's not recommended because the performance, for example when running jobs can be very poor. Here is one such experience.
Created ‎02-01-2016 09:42 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Of course that possible. This doc could help you Topology scripts are used by Hadoop to determine the rack location of nodes
Created ‎02-01-2016 10:04 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @Kabirdas B K yes that's possible, you can just add remote nodes like local nodes, and yes you can set rack awareness, but it's not recommended because the performance, for example when running jobs can be very poor. Here is one such experience.
Created ‎02-01-2016 10:44 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This! You can but you shouldn't. Cross-datacenter hadoop clusters do not perform. Don't do it.
Use data replication software like DistCP or Falcon to sync clusters in different data centers instead.
Created ‎02-01-2016 11:10 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Do not do it! @Kabirdas B K
Created ‎02-01-2016 11:15 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I don't recommend doing this. It's possible but not the best practice. You can setup DR using Falcon or WanDisco.
