Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Who agreed with this solution

avatar
Expert Contributor

Thanks ... here's what happened:

 

  • LinuxContainerExecutor was a dead-end.  Set up all the config as recommended - Yarn switched to user 'nobody', but not the WF owner ('myao').  This put MR out of order cluster-wide - even Hive CLI stopped working.  yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users = false has no effect ... I was kinda expecting that given that YARN-2424 was for Hadoop 2.6.0, and our CDH 5.2.0 has 2.5.0.
  • Your workaround #2, set HADOOP_USER_NAME=myao in shell action, succeeded.  I was able to run "hive -e <query>" and capture the result.

 

To summarize, the minimal manual config to get shell action working normally as workflow owner for us:

  • HADOOP_CONF_DIR=/etc/hive/conf
  • HIVE_CONF_DIR=/etc/hive/conf
  • HADOOP_USER_NAME=${wf:user()}

 

Appreciate your advices.  Please feel free to comment and share your experiences!

 

 

View solution in original post

Who agreed with this solution