And the solution appears to be we had some files .../hadoop/yarn/local/usercache/hive/filecache with odd permissions. Flushing that directory on the data nodes and the query runs. ... View more