Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this solution

avatar
Expert Contributor

Thanks ... here's what happened:

 

  • LinuxContainerExecutor was a dead-end.  Set up all the config as recommended - Yarn switched to user 'nobody', but not the WF owner ('myao').  This put MR out of order cluster-wide - even Hive CLI stopped working.  yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users = false has no effect ... I was kinda expecting that given that YARN-2424 was for Hadoop 2.6.0, and our CDH 5.2.0 has 2.5.0.
  • Your workaround #2, set HADOOP_USER_NAME=myao in shell action, succeeded.  I was able to run "hive -e <query>" and capture the result.

 

To summarize, the minimal manual config to get shell action working normally as workflow owner for us:

  • HADOOP_CONF_DIR=/etc/hive/conf
  • HIVE_CONF_DIR=/etc/hive/conf
  • HADOOP_USER_NAME=${wf:user()}

 

Appreciate your advices.  Please feel free to comment and share your experiences!

 

 

View solution in original post

Who agreed with this solution