Hive query failing with: java.io.IOException: Previous writer likely failed to write ... Failing because I am unlikely to write too.

[#1]

We recently upgraded our cluster from HDP2.3 to 2.4 and have started seeing this error:

java.io.IOException: Previous writer likely failed to write hdfs://nameservice1/tmp/hive/cloudfeeds/_tez_session_dir/419d4163-bc94-45ea-99c4-c82148f8589b/auxlib. Failing because I am unlikely to write too.

We already have the HIVE_AUX_JARS_PATH set in hive-env.sh as suggested in other posts (see below).

Any other ideas what might be causing this error?

if [ "${HIVE_AUX_JARS_PATH}" != "" ]; then
  if [ -f "${HIVE_AUX_JARS_PATH}" ]; then
    export HIVE_AUX_JARS_PATH=${HIVE_AUX_JARS_PATH}
  elif [ -d "/usr/hdp/current/hive-webhcat/share/hcatalog" ]; then
    export HIVE_AUX_JARS_PATH=/usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar
  fi
elif [ -d "/usr/hdp/current/hive-webhcat/share/hcatalog" ]; then
  export HIVE_AUX_JARS_PATH=/usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar
fi

[#2 · Accepted solution]

We finally traced the problem down to a configuration error. Our hive-site.xml file contained a hive.aux.jars.path property that was set to a path that didn't exist. Once we removed this property and restarted hive and oozie, oozie was again able to start our hive jobs.

[#3]

I had the same problem. Check that hive-env.sh exist and has the fix https://issues.apache.org/jira/browse/HIVE-16398