- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to improve the performance of NFS Server?
- Labels:
-
Apache Hadoop
Created 10-07-2015 03:00 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
A customer is experiencing slowness with NFS when copying large files through NFS.
Question:
1) Is NFS recommended for large files?
2) What performance tuning options are available for NFS Server?
Created 10-07-2015 06:32 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did the customer use the 'sync' option while mounting the share on the NFS client? Large file transfers (few GBs and larger) are slow without 'sync' and often stall completely.
'sync' ensures that the client will not reorder writes. Reordered writes force the NFS gateway to buffer data since HDFS only supports sequential writes (appends). You can also try increasing 'nfs.rtmax' and 'nfs.wtmax' in the NFS gateway configuration as recommended in the same link.
It looks like we are missing NFS Gateway documentation in the the HDP docs. I'll make sure we get that updated.
Created 10-07-2015 03:00 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Are you talking about HDFS NFS Gateway ?.
Created 10-07-2015 03:43 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes. The HDFS gateway
Created 10-07-2015 06:32 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did the customer use the 'sync' option while mounting the share on the NFS client? Large file transfers (few GBs and larger) are slow without 'sync' and often stall completely.
'sync' ensures that the client will not reorder writes. Reordered writes force the NFS gateway to buffer data since HDFS only supports sequential writes (appends). You can also try increasing 'nfs.rtmax' and 'nfs.wtmax' in the NFS gateway configuration as recommended in the same link.
It looks like we are missing NFS Gateway documentation in the the HDP docs. I'll make sure we get that updated.
