question Permission Denied for user while creating a hive table using sqoop in Oozie workflow in Archives of Support Questions (Read Only)

Permission Denied for user while creating a hive table using sqoop in Oozie workflow

vinayakagrawal8 — Thu, 21 Jan 2016 04:08:58 GMT

Hi,

Environment : HDP 2.3.4, 3 Nodes, 32GB ram, redhat

I am running a sqoop job via Hue-oozie. The job is to import data from Mysql into Hive.

As I understand it, Sqoop first imports the data from MYSQL using JDBC and writes that into HDFS.

Then Sqoop calls Hive to create a table and move the same data into Hive warehouse to create a hive internal table.

So far, the part where Sqoop imports data from MYSQL completes successfully, BUT the task to create Hive table keeps on failing.

Below is the snippet for the error.

26225 [Thread-28] INFO  org.apache.sqoop.hive.HiveImport  - org.apache.hadoop.security.AccessControlException: Permission denied: user=urep, access=EXECUTE, inode="/tmp/hive/yarn/_tez_session_dir/3007739a-c7fe-4730-a360-ba304646bc3b/hive-hcatalog-core.jar":yarn:hdfs:drwx------
2016-01-20 09:48:21,069 INFO  [Thread-28] hive.HiveImport (LoggingAsyncSink.java:run(85)) - org.apache.hadoop.security.AccessControlException: Permission denied: user=urep, access=EXECUTE, inode="/tmp/hive/yarn/_tez_session_dir/3007739a-c7fe-4730-a360-ba304646bc3b/hive-hcatalog-core.jar":yarn:hdfs:drwx------

What I HAVE TRIED so far:

1. https://community.hortonworks.com/questions/8158/permission-denied-for-userhive-on-load-data-inpath.html

I have given 777 permission to /tmp/hive/yarn/_tez_session_dir but did not help.

2. https://community.hortonworks.com/questions/7642/oozie-shell-action-permission-denied-userxyz-acces.html

I added the hadoop user environment variable either but that didnt work.

Please help

Re: Permission Denied for user while creating a hive table using sqoop in Oozie workflow

aervits — Thu, 21 Jan 2016 04:13:43 GMT

@Vinayak Agrawal try sudo -u hdfs hdfs dfs -chmod -R 777 /tmp/hive/yarn

Re: Permission Denied for user while creating a hive table using sqoop in Oozie workflow

vinayakagrawal8 — Mon, 19 Aug 2019 11:55:05 GMT

Artem Thanks for your reply. I have given 777 recursively to /tmp/hive/yarn which did not help

Re: Permission Denied for user while creating a hive table using sqoop in Oozie workflow

aervits — Thu, 21 Jan 2016 04:26:11 GMT

doesn't look like write permission for other took effect if you say 777 was applied. Is user urep part of any group or part of other?

Re: Permission Denied for user while creating a hive table using sqoop in Oozie workflow

SK1 — Fri, 22 Jan 2016 02:43:27 GMT

@Vinayak : What is your /tmp dir permission and if it is not 777 then please make it to 777.

Re: Permission Denied for user while creating a hive table using sqoop in Oozie workflow

aervits — Wed, 03 Feb 2016 05:23:09 GMT

@Vinayak Agrawal are you still having problems with this? Can you provide your own solution or accept best answer?

Re: Permission Denied for user while creating a hive table using sqoop in Oozie workflow

Shelton — Wed, 03 Feb 2016 05:38:13 GMT

@Vinayak Agrawal Are you running the sqoop job as user yarn? if not the user running the sqoop job should have the appropriate permissions eg 777 /tmp/hive/yarn/_tez_session_dir/xxxxx

Re: Permission Denied for user while creating a hive table using sqoop in Oozie workflow

sim6 — Sat, 14 May 2016 20:54:14 GMT

@Vinayak Agrawal: Mind sharing your solution? I have exactly the same problem

Re: Permission Denied for user while creating a hive table using sqoop in Oozie workflow

nsabharwal — Sat, 14 May 2016 21:09:02 GMT

@Vinayak Agrawal

try this

try modifying the workflow.xml and add

<exec>${execFile}</exec> 
 <env-var>HADOOP_USER_NAME=${wf:user()}</env-var> 
<file>${execPath}#${execFile}</file>

Re: Permission Denied for user while creating a hive table using sqoop in Oozie workflow

bigdataam09 — Wed, 08 Jun 2016 04:44:17 GMT

I resolved this issue by copying the hive-hcatalog-core.jar file directly under lib folder of the workflow

Re: Permission Denied for user while creating a hive table using sqoop in Oozie workflow

ramcharantej — Fri, 06 Jan 2017 15:57:18 GMT

Hi Amit, Can you please let us know the location that you have added this hive-hcatalog-core.jar file, we are facing similar issue now.

PS: We are trying to run hive query in Shell Script in Oozie hue.

Regards,

Ram