@Mon key In HDFS, reads normally go through the DataNode. Thus, when the client asks the DataNode to read a file, the DataNode reads that file off of the disk and sends the data to the client over a TCP socket. So-called “short-circuit” reads bypass the DataNode, allowing the client to read the file directly. Obviously, this is only possible in cases where the client is co-located with the data. Short-circuit reads provide a substantial performance boost to many applications. To configure short-circuit local reads, you must enable libhadoop.so . See Native Libraries for details on enabling this library. Windows is not a supported OS. You need to turn off this feature and re-execute your job.
... View more
