question Permission Denied using Sqoop from MySQL on sandbox in Archives of Support Questions (Read Only)

Permission Denied using Sqoop from MySQL on sandbox

dandreae — Fri, 16 Sep 2022 09:57:32 GMT

I am using Hortonworks Sandbox (a clean setup), and created a simple table in mysql which I want to sqoop into hdfs.

I'm used the MySql that was already installed on the sandbox, by running 'mysql' in the shell.

My first attempt at sqoop was:

sqoop import --connect jdbc:mysql://localhost/test --table mytable

This gave "..access denied to localhost.." error, so based on an answer another post, and finding the MySQL port using "show variables like 'port'" within MySQL, I then tried:

sqoop import --connect jdbc:mysql://sandbox.hortonworks.com:3306/test --username root --table mytable

This gave errors about ".. streaming result sets are open...", which was solved by another post by adding the 'driver' argument as follows:

sqoop import --connect jdbc:mysql://sandbox.hortonworks.com:3306/test --username root --table mytable --driver com.mysql.jdbc.Driver

It now gets a bit further, but now fails with a RemoteException: ...Permission denied: user=root, access=WRITE, inode="user/root/.staging":hdfs:hdfs:drwxr-xr-x

That's as far as I've got. Am I doing something wrong, or do I need to set up some access permissions somewhere? I would've expected that many people have been sqooping from MySql without having to change the set up, so is my sqoop command incorrect?

Re: Permission Denied using Sqoop from MySQL on sandbox

nsabharwal — Fri, 15 Jan 2016 05:03:36 GMT

@David Andreae

Its solely permission issues.

1) Setup mysql permission for root user

2) hdfs dfs -chown -R root:hadoop /user/root

Re: Permission Denied using Sqoop from MySQL on sandbox

dandreae — Fri, 15 Jan 2016 19:06:42 GMT

Thanks Neeraj,

But it seems there is no 'root' directory under '/user' in my HDFS anyway. So running the 'chown' command you gave above it says "No such file or directory".

When I try it on just the '/user' directory (ie. .. -chown .. /user) it says "Permission denied: user=root is not the owner of inode=user

So I tried changing the sqoop target-dir to a folder under /tmp, but it still complains about denied access to '/user/root'. ??

I guess I'm not understanding how the hadoop hdfs folders, owners, permissions etc fit together, and cannot find a clear summary of it anywhere.

Re: Permission Denied using Sqoop from MySQL on sandbox

amcbarnett — Sat, 16 Jan 2016 00:19:15 GMT

sudo su - hdfs
hdfs dfs -mkdir /user/root
hdfs dfs -chown -R root:hdfs /user/root
exit

Re: Permission Denied using Sqoop from MySQL on sandbox

dandreae — Sat, 16 Jan 2016 00:24:52 GMT

Thanks Ancil, that works, although I actually did it slightly differently based on another post, using '..-R root:root..' rather than ..-R root:hdfs..'. I guess that means I've changed the group to 'root' when I could've left it as 'hdfs'. Didn't seem to matter, so far.