I use the same command and have no issues. According to logs: Caused by: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local directory for output/attempt_1523546159827_0013_r_000000_0/map_0.out So, I would guess that you csv is too big and when the reducer tries to load it, there is no sufficient space in local dirs of YARN nodemanager.   Can you try set more reducers by using : --reducers 4 or more (based on your partitions and the csv size). You can also set more mappers, but based on log the reducer is suffering. More details: https://www.cloudera.com/documentation/enterprise/5-13-x/topics/search_mapreduceindexertool.html#concept_pjs_3sd_3v ... View more

This is a mis-leading of the "free" output. The first line (starting with "Mem") displays that you have 62G of memory and 56G are used. This memory is used but not from procesess. At the end of the line, you will see a number of 39G  cached. In few words, Linux uses a part of free RAM to store data from files used often, in order to save some interactions with the hard disk. Once an application request memory and there is no "free", Linux automatically drops these caches. You cannot turn this feature off. The only thing you can do is just drop the current cached data, but Linux will store something the very next second. In any case, when the output of "free" is similar to the one you provided, you should always refer to the second line "-/+ buffers/cache: 16G 49G" This is the real status, which show "16G" used and "49G" free.   Finally, CM displays the disk and memory usage of the host (in Hosts view) regardless of what process is using it. It is the same output as "free". ... View more

The error complains about the value of "hadoop.security.authentication". You have set it to "Kerberos" while the accepted values are "simple" and "kerberos" (all letters in lowercase). ... View more