Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

When execute "CREATE TABLE" with location by impala will cause privilege error.

Solved Go to solution

When execute "CREATE TABLE" with location by impala will cause privilege error.

Explorer

Hi,

 

I enabled sentry for impala and sync to HDFS.

 

Mostly working correctly, but when execute "CREATE TABLE" with location by impala will cause unexpected privilege error.
Despite having authority to the specified location.

 

CDH Settings

hadoop.security.group.mapping: ShellBasedUnixGroupsMapping
hadoop.security.authentication: simple
hive.sentry.provider: HadoopGroupResourceAuthorizationProvider

Authentication is all disabled for hdfs/hive/impala.

 

 Details are described below.

 

1. The first case is when location for table directory is not specified.

 

[root@hostname ~]# su test_user1
[test_user1@hostname ~]$ impala-shell

[hostname.example.com:21000] > CREATE EXTERNAL TABLE `test_db`.`test_table1` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile TBLPROPERTIES("skip.header.line.count" = "1");
Fetched 0 row(s) in 0.50s

 

This worked.

 

Check the authority of the created impala table directory. 

[root@hostname ~]# hdfs dfs -getfacl /user/hive/warehouse/test_db.db/test_table1

# file: /user/hive/warehouse/trial_f2042910.db/test_table1
# owner: hive
# group: hive
user:hive:rwx
user:test_user1:rwx
group:hive:rwx
group:test_group1:rwx
mask::rwx
other::--x

 

The all authority is given to "test_user1".

 

 

2. The next case is when location for table directory is specified.

 

[root@hostname ~]# hdfs dfs -getfacl /user/hive/warehouse/test_db.db/test_table2
# file: /user/hive/warehouse/test_db.db/test_table2 # owner: hive # group: hive user:hive:rwx user:test_user2:rwx group:hive:rwx group:test_group2:rwx mask::rwx other::--x

 

The all authority is given to "test_user2".

 

[root@hostname ~]# su test_user2
[test_user2@hostname ~]$ impala-shell

[hostname.example.com:21000] > CREATE EXTERNAL TABLE `test_db`.`test_table2` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile LOCATION '/user/hive/warehouse/test_db.db/test_table2' TBLPROPERTIES("skip.header.line.count" = "1");

ERROR: AuthorizationException: User 'test_user2' does not have privileges to access: hdfs://hostname.example.com:8020/user/hive/warehouse/test_db.db/test_table2

 

This not worked. 

 

why?

 

By the way, with the hdfs command can write without problems.

 

[root@hostname ~] su test_user2
[test_user2@hostname ~]$ hdfs dfs -put test.csv /user/hive/warehouse/test_db.db/test_table2/
=> success

 

 

The clues are that there is a difference in the impala deamon log.

 

1. The first case is when location for table directory is not specified.

 

I1130 19:00:53.146760  3080 impala-hs2-server.cc:418] ExecuteStatement(): request=TExecuteStatementReq {
  01: sessionHandle (struct) = TSessionHandle {
    01: sessionId (struct) = THandleIdentifier {
      01: guid (string) = ">\xfa\xb2|/\xe3J\xde\x978>\xfb\xf9\xc9k\x13",
      02: secret (string) = "p\"a\xee\xd4\xc4G\x1d\x9aOV\xbe6\x17\xa6\x8b",
    },
  },
  02: statement (string) = "CREATE EXTERNAL TABLE `test_db`.`test_table1` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile TBLPROPERTIES(\"skip.header.line.count\" = \"1\")",
  03: confOverlay (map) = map<string,string>[2] {
    "QUERY_TIMEOUT_S" -> "600",
    "impala.resultset.cache.size" -> "100000",
  },
  04: runAsync (bool) = true,
}
.
.

 

2. The next case is when location for table directory is specified.

 

I1130 19:08:29.901100 18617 impala-beeswax-server.cc:52] query(): query=CREATE EXTERNAL TABLE `test_db`.`test_table2` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile LOCATION '/user/hive/warehouse/test_db.db/test_table2' TBLPROPERTIES("skip.header.line.count" = "1")
I1130 19:08:29.901142 18617 impala-beeswax-server.cc:426] query: Query {
  01: query (string) = "CREATE EXTERNAL [...](259)",
  03: configuration (list) = list<string>[0] {
  },
  04: hadoop_user (string) = "test_user2",
}
.
.

 

 

When location is not specified, the query is executed with ExecuteStatement() method of impala-hs2-server.cc.

But location is specified, the query is executed with query() method of impala-beeswax-server.cc.

 

 

Do you know what is wrong?

Is this a bug?

 

 

Thank you in advance.

uma66. 

 

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: When execute "CREATE TABLE" with location by impala will cause privilege error.

Explorer

The problem was solved.

 

I had granted privileges with "hdfs dfs -setfacl" command, but I needed to grant privileges with "GRANT ON URI" command from impala.

 

 

 

2 REPLIES 2
Highlighted

Re: When execute "CREATE TABLE" with location by impala will cause privilege error.

Explorer

The problem was solved.

 

I had granted privileges with "hdfs dfs -setfacl" command, but I needed to grant privileges with "GRANT ON URI" command from impala.

 

 

 

Re: When execute "CREATE TABLE" with location by impala will cause privilege error.

Community Manager

Congratulations on solving your issue and thank you for marking the solution to assist others in a similar situation. :)



Cy Jervis, Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:
Community Guidelines
How to use the forum