Thanks ... here's what happened:
- LinuxContainerExecutor was a dead-end. Set up all the config as recommended - Yarn switched to user 'nobody', but not the WF owner ('myao'). This put MR out of order cluster-wide - even Hive CLI stopped working. yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users = false has no effect ... I was kinda expecting that given that YARN-2424 was for Hadoop 2.6.0, and our CDH 5.2.0 has 2.5.0.
- Your workaround #2, set HADOOP_USER_NAME=myao in shell action, succeeded. I was able to run "hive -e <query>" and capture the result.
To summarize, the minimal manual config to get shell action working normally as workflow owner for us:
- HADOOP_CONF_DIR=/etc/hive/conf
- HIVE_CONF_DIR=/etc/hive/conf
- HADOOP_USER_NAME=${wf:user()}
Appreciate your advices. Please feel free to comment and share your experiences!