When execute "CREATE TABLE" with location by impala will cause privilege error.

uma66 — Fri, 16 Sep 2022 13:56:52 GMT

Hi,

I enabled sentry for impala and sync to HDFS.

Mostly working correctly, but when execute "CREATE TABLE" with location by impala will cause unexpected privilege error.
Despite having authority to the specified location.

CDH Settings

hadoop.security.group.mapping: ShellBasedUnixGroupsMapping
hadoop.security.authentication: simple
hive.sentry.provider: HadoopGroupResourceAuthorizationProvider

Authentication is all disabled for hdfs/hive/impala.

Details are described below.

1. The first case is when location for table directory is not specified.

[root@hostname ~]# su test_user1
[test_user1@hostname ~]$ impala-shell

[hostname.example.com:21000] > CREATE EXTERNAL TABLE `test_db`.`test_table1` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile TBLPROPERTIES("skip.header.line.count" = "1");
Fetched 0 row(s) in 0.50s

This worked.

Check the authority of the created impala table directory.

[root@hostname ~]# hdfs dfs -getfacl /user/hive/warehouse/test_db.db/test_table1

# file: /user/hive/warehouse/trial_f2042910.db/test_table1
# owner: hive
# group: hive
user:hive:rwx
user:test_user1:rwx
group:hive:rwx
group:test_group1:rwx
mask::rwx
other::--x

The all authority is given to "test_user1".

2. The next case is when location for table directory is specified.

[root@hostname ~]# hdfs dfs -getfacl /user/hive/warehouse/test_db.db/test_table2

# file: /user/hive/warehouse/test_db.db/test_table2
# owner: hive
# group: hive
user:hive:rwx
user:test_user2:rwx
group:hive:rwx
group:test_group2:rwx
mask::rwx
other::--x

The all authority is given to "test_user2".

[root@hostname ~]# su test_user2
[test_user2@hostname ~]$ impala-shell

[hostname.example.com:21000] > CREATE EXTERNAL TABLE `test_db`.`test_table2` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile LOCATION '/user/hive/warehouse/test_db.db/test_table2' TBLPROPERTIES("skip.header.line.count" = "1");

ERROR: AuthorizationException: User 'test_user2' does not have privileges to access: hdfs://hostname.example.com:8020/user/hive/warehouse/test_db.db/test_table2

This not worked.

why?

By the way, with the hdfs command can write without problems.

[root@hostname ~] su test_user2
[test_user2@hostname ~]$ hdfs dfs -put test.csv /user/hive/warehouse/test_db.db/test_table2/
=> success

The clues are that there is a difference in the impala deamon log.

1. The first case is when location for table directory is not specified.

I1130 19:00:53.146760  3080 impala-hs2-server.cc:418] ExecuteStatement(): request=TExecuteStatementReq {
  01: sessionHandle (struct) = TSessionHandle {
    01: sessionId (struct) = THandleIdentifier {
      01: guid (string) = ">\xfa\xb2|/\xe3J\xde\x978>\xfb\xf9\xc9k\x13",
      02: secret (string) = "p\"a\xee\xd4\xc4G\x1d\x9aOV\xbe6\x17\xa6\x8b",
    },
  },
  02: statement (string) = "CREATE EXTERNAL TABLE `test_db`.`test_table1` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile TBLPROPERTIES(\"skip.header.line.count\" = \"1\")",
  03: confOverlay (map) = map<string,string>[2] {
    "QUERY_TIMEOUT_S" -> "600",
    "impala.resultset.cache.size" -> "100000",
  },
  04: runAsync (bool) = true,
}
.
.

2. The next case is when location for table directory is specified.

I1130 19:08:29.901100 18617 impala-beeswax-server.cc:52] query(): query=CREATE EXTERNAL TABLE `test_db`.`test_table2` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile LOCATION '/user/hive/warehouse/test_db.db/test_table2' TBLPROPERTIES("skip.header.line.count" = "1")
I1130 19:08:29.901142 18617 impala-beeswax-server.cc:426] query: Query {
  01: query (string) = "CREATE EXTERNAL [...](259)",
  03: configuration (list) = list<string>[0] {
  },
  04: hadoop_user (string) = "test_user2",
}
.
.

When location is not specified, the query is executed with ExecuteStatement() method of impala-hs2-server.cc.

But location is specified, the query is executed with query() method of impala-beeswax-server.cc.

Do you know what is wrong?

Is this a bug?

Thank you in advance.

uma66.

Re: When execute "CREATE TABLE" with location by impala will cause privilege error.

uma66 — Mon, 10 Dec 2018 06:37:59 GMT

The problem was solved.

I had granted privileges with "hdfs dfs -setfacl" command, but I needed to grant privileges with "GRANT ON URI" command from impala.

Re: When execute "CREATE TABLE" with location by impala will cause privilege error.

cjervis — Mon, 10 Dec 2018 14:38:40 GMT

Congratulations on solving your issue and thank you for marking the solution to assist others in a similar situation. 🙂

question Re: When execute "CREATE TABLE" with location by impala will cause privilege error. in Archives of Support Questions (Read Only)

When execute "CREATE TABLE" with location by impala will cause privilege error.

Re: When execute "CREATE TABLE" with location by impala will cause privilege error.

Re: When execute "CREATE TABLE" with location by impala will cause privilege error.