Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Impala-shell thinks I'm root when I'm not

Impala-shell thinks I'm root when I'm not

New Contributor

Hello all

 

I am getting some strange errors on a new installation of CDH4 w/ Impala.

 

Working through the tutorial at 

 

http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Im...

 

It's all going ok, until I hit the 


CREATE TABLE tab3 ( id INT, col_1 BOOLEAN, col_2 DOUBLE, month INT, day INT ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';

 

Statement. This fails with:

 

Query: create TABLE tab_3
(
id INT,
col_1 BOOLEAN,
col_2 DOUBLE,
month INT,
day INT
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
ERROR: MetaException: Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=root, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

 

What puzzles me, is where the "user=root" comes from. I am logged on to the system as the os user tutorial. No connection to root whatsoever.

 

 

 


[happy01.camper:21000] > Goodbye
[tutorial@happy01 ~]$ id -a
uid=501(tutorial) gid=501(tutorial) groups=501(tutorial)
[tutorial@happy01 ~]$

 

-bash-4.1$ hadoop fs -ls /user

Found 1 items
drwxr-xr-x - tutorial tutorial 0 2013-09-11 00:17 /user/tutorial
-bash-4.1$ hadoop fs -ls /user/tutorial
Found 4 items
drwxr-xr-x - tutorial tutorial 0 2013-09-11 00:17 /user/tutorial/tab1
drwxr-xr-x - tutorial tutorial 0 2013-09-11 00:11 /user/tutorial/tab_1
drwxr-xr-x - tutorial tutorial 0 2013-09-11 00:11 /user/tutorial/tab_2
drwxr-xr-x - tutorial tutorial 0 2013-09-11 00:11 /user/tutorial/tab_3

 

Any ideas why this is happening ?

 

Cheers

 

Stefan

 

PS: Asking a new un-registered user to "Post a question" and then redirecting him to a registration screen and just throwing away the data he entered making a new post is not very nice :)

7 REPLIES 7
Highlighted

Re: Impala-shell thinks I'm root when I'm not

Cloudera Employee

I think you may be missing some details of the tutorial -- you will want to use "external" tables and specify the path to the files.

 

-- The EXTERNAL clause means the data is located outside the central location for Impala data files
-- and is preserved when the associated Impala table is dropped. We expect the data to already
-- exist in the directory specified by the LOCATION clause.

 

This should do the job:

 

CREATE EXTERNAL TABLE tab3 ( id INT, col_1 BOOLEAN, col_2 DOUBLE, month INT, day INT ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION '/user/tutorial/tab_3';

 

Re: Impala-shell thinks I'm root when I'm not

New Contributor

Hi Greg

 

The first two CREATE TABLE statements in the aforementioned tutorial are using the EXTERNAL clause. tab3 however, is not.

 

Is this a mistake in the tutorial?

 

But the same problem reproduces by issuing the most basic

 

[happy01.camper:21000] > create table t (x int);
Query: create table t (x int)
ERROR: MetaException: Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=root, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

 

 

Stefan

Re: Impala-shell thinks I'm root when I'm not

Cloudera Employee

It's not a mistake in the tutorial to have TAB3 as a managed table, it's there to show copying from one table to another:

 

-- Leaving out the EXTERNAL clause means the data will be managed
-- in the central Impala data directory tree. Rather than reading
-- existing data files when the table is created, we load the
-- data after creating the table.
CREATE TABLE tab3 ...

The bit that is unnecessary is to make a sample_data/tab3 directory in HDFS, since no data file is ever put there. I'll take out that last argument:

 

hdfs dfs -mkdir -p /user/cloudera/sample_data/tab1 /user/cloudera/sample_data/tab2 /user/cloudera/sample_data/tab3

Re: Impala-shell thinks I'm root when I'm not

New Contributor

So did anyone figure this out?  I am having the same problem.  I changed so many permissions at this point that my head is spinning out of control.  Did anyone get this resolved, if so please help me out?

 

My files are in a directory /user/data/table1

I chmod-ed the hell out of everything to 777, made the user and group impala:impala throughout the whole directory hdfs directory /user and down, but am getting the same annoying error.  What am I doing wrong?  Is there some other magic directory I need to create that the error messaging is neglecting to tell me about?

 

Help?

 

THE ERROR:

 

[localhost.localdomain:21000] > create external table table1 ( id int, name string, age int ) row format delimited fields terminated by ',' location '/user/data/table1';
Query: create external table table1 ( id int, name string, age int ) row format delimited fields terminated by ',' location '/user/data/table1'
ERROR: MetaException: Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=impala, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x


MY DATA FILES:

 

[root@localhost /]# hdfs dfs -ls /user/data/table1/
Found 1 items
-rwxrwxrwx   1 impala impala         50 2013-10-19 09:02 /user/data/table1/data1.dat

Re: Impala-shell thinks I'm root when I'm not

New Contributor

Please folks: I'm a new user and I like the Manager and all the tools very much. However, I'm getting the same error when I try to CREATE TABLE tab3:

 

ERROR: MetaException: Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=impala, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x

But in my case the user that the command line picked up was impala

 

What puzzles me, is where the "user=root" comes from. I am logged on to the system as the os user tutorial. No connection to root whatsoever.

Same here. I had a look at impala-shell.py, but I couldn't find where this comes from.

 

Help? :heart:

Re: Impala-shell thinks I'm root when I'm not

Contributor
1. All Impala queries run as user "Impala", thats where the user came from.
For the OP, I'm guessing he installed Impala as root.

2. Create table query (for non-external tables) creates a table in /user/hive/warehouse directory. Impala should be able to write there.

Hope this helps.

Re: Impala-shell thinks I'm root when I'm not

Cloudera Employee

This sounds like an issue that was discussed here on the mailing list:

 

https://groups.google.com/a/cloudera.org/forum/#!searchin/impala-user/issues$20when$20hdfs/impala-us...

 

The setting of the hive.metastore.warehouse.dir property can be different if you look in the Hive shell or in Cloudera Manager dialogs. The directory tree that Impala uses (shared with Hive) needs to have ownership hive:hive. If some nodes have outdated configuration, you might need to specify them as 'gateways' (the gateway role) and 'deploy client configuration' to get the latest hive.metastore.warehouse.dir property out to them:

 

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/4.5.3/Cloudera-Manager-Enterpr...

 

 

Let us know if that helps!

 

John