Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st. We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here. Need help or have questions? Drop us a line at [email protected]
Created on 06-30-201811:46 AM - edited 09-16-202201:43 AM
Hadoop relies heavily on DNS, and as such performs many DNS lookups during normal operation. To reduce the load on your DNS infrastructure, it's highly recommended to use the Name Service Caching Daemon (NSCD) on cluster nodes running Linux or any other dns caching mechanism ( i.e dnsmasq ).
This daemon will cache host, user, and group lookups and provide better resolution performance, and reduced load on DNS infrastructure.
Default cache size and TTL value are good enough to reduce the significant load. However, you might need to tweak this as per your environment.