Support Questions
Find answers, ask questions, and share your expertise

Permission Denied using Sqoop from MySQL on sandbox

Solved Go to solution
Highlighted

Permission Denied using Sqoop from MySQL on sandbox

Explorer

I am using Hortonworks Sandbox (a clean setup), and created a simple table in mysql which I want to sqoop into hdfs.

I'm used the MySql that was already installed on the sandbox, by running 'mysql' in the shell.

My first attempt at sqoop was:

sqoop import --connect jdbc:mysql://localhost/test --table mytable

This gave "..access denied to localhost.." error, so based on an answer another post, and finding the MySQL port using "show variables like 'port'" within MySQL, I then tried:

sqoop import --connect jdbc:mysql://sandbox.hortonworks.com:3306/test --username root --table mytable

This gave errors about ".. streaming result sets are open...", which was solved by another post by adding the 'driver' argument as follows:

sqoop import --connect jdbc:mysql://sandbox.hortonworks.com:3306/test --username root --table mytable --driver com.mysql.jdbc.Driver

It now gets a bit further, but now fails with a RemoteException: ...Permission denied: user=root, access=WRITE, inode="user/root/.staging":hdfs:hdfs:drwxr-xr-x

That's as far as I've got. Am I doing something wrong, or do I need to set up some access permissions somewhere? I would've expected that many people have been sqooping from MySql without having to change the set up, so is my sqoop command incorrect?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Permission Denied using Sqoop from MySQL on sandbox

sudo su - hdfs
hdfs dfs -mkdir /user/root
hdfs dfs -chown -R root:hdfs /user/root
exit

View solution in original post

4 REPLIES 4

Re: Permission Denied using Sqoop from MySQL on sandbox

@David Andreae

Its solely permission issues.

1) Setup mysql permission for root user

2) hdfs dfs -chown -R root:hadoop /user/root

Highlighted

Re: Permission Denied using Sqoop from MySQL on sandbox

Explorer

Thanks Neeraj,

But it seems there is no 'root' directory under '/user' in my HDFS anyway. So running the 'chown' command you gave above it says "No such file or directory".

When I try it on just the '/user' directory (ie. .. -chown .. /user) it says "Permission denied: user=root is not the owner of inode=user

So I tried changing the sqoop target-dir to a folder under /tmp, but it still complains about denied access to '/user/root'. ??

I guess I'm not understanding how the hadoop hdfs folders, owners, permissions etc fit together, and cannot find a clear summary of it anywhere.

Highlighted

Re: Permission Denied using Sqoop from MySQL on sandbox

sudo su - hdfs
hdfs dfs -mkdir /user/root
hdfs dfs -chown -R root:hdfs /user/root
exit

View solution in original post

Highlighted

Re: Permission Denied using Sqoop from MySQL on sandbox

Explorer

Thanks Ancil, that works, although I actually did it slightly differently based on another post, using '..-R root:root..' rather than ..-R root:hdfs..'. I guess that means I've changed the group to 'root' when I could've left it as 'hdfs'. Didn't seem to matter, so far.