Hi,
I enabled sentry for impala and sync to HDFS.
Mostly working correctly, but when execute "CREATE TABLE" with location by impala will cause unexpected privilege error.
Despite having authority to the specified location.
CDH Settings
hadoop.security.group.mapping: ShellBasedUnixGroupsMapping
hadoop.security.authentication: simple
hive.sentry.provider: HadoopGroupResourceAuthorizationProvider
Authentication is all disabled for hdfs/hive/impala.
Details are described below.
1. The first case is when location for table directory is not specified.
[root@hostname ~]# su test_user1
[test_user1@hostname ~]$ impala-shell
[hostname.example.com:21000] > CREATE EXTERNAL TABLE `test_db`.`test_table1` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile TBLPROPERTIES("skip.header.line.count" = "1");
Fetched 0 row(s) in 0.50s
This worked.
Check the authority of the created impala table directory.
[root@hostname ~]# hdfs dfs -getfacl /user/hive/warehouse/test_db.db/test_table1
# file: /user/hive/warehouse/trial_f2042910.db/test_table1
# owner: hive
# group: hive
user:hive:rwx
user:test_user1:rwx
group:hive:rwx
group:test_group1:rwx
mask::rwx
other::--x
The all authority is given to "test_user1".
2. The next case is when location for table directory is specified.
[root@hostname ~]# hdfs dfs -getfacl /user/hive/warehouse/test_db.db/test_table2
# file: /user/hive/warehouse/test_db.db/test_table2
# owner: hive
# group: hive
user:hive:rwx
user:test_user2:rwx
group:hive:rwx
group:test_group2:rwx
mask::rwx
other::--x
The all authority is given to "test_user2".
[root@hostname ~]# su test_user2
[test_user2@hostname ~]$ impala-shell
[hostname.example.com:21000] > CREATE EXTERNAL TABLE `test_db`.`test_table2` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile LOCATION '/user/hive/warehouse/test_db.db/test_table2' TBLPROPERTIES("skip.header.line.count" = "1");
ERROR: AuthorizationException: User 'test_user2' does not have privileges to access: hdfs://hostname.example.com:8020/user/hive/warehouse/test_db.db/test_table2
This not worked.
why?
By the way, with the hdfs command can write without problems.
[root@hostname ~] su test_user2
[test_user2@hostname ~]$ hdfs dfs -put test.csv /user/hive/warehouse/test_db.db/test_table2/
=> success
The clues are that there is a difference in the impala deamon log.
1. The first case is when location for table directory is not specified.
I1130 19:00:53.146760 3080 impala-hs2-server.cc:418] ExecuteStatement(): request=TExecuteStatementReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = ">\xfa\xb2|/\xe3J\xde\x978>\xfb\xf9\xc9k\x13",
02: secret (string) = "p\"a\xee\xd4\xc4G\x1d\x9aOV\xbe6\x17\xa6\x8b",
},
},
02: statement (string) = "CREATE EXTERNAL TABLE `test_db`.`test_table1` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile TBLPROPERTIES(\"skip.header.line.count\" = \"1\")",
03: confOverlay (map) = map<string,string>[2] {
"QUERY_TIMEOUT_S" -> "600",
"impala.resultset.cache.size" -> "100000",
},
04: runAsync (bool) = true,
}
.
.
2. The next case is when location for table directory is specified.
I1130 19:08:29.901100 18617 impala-beeswax-server.cc:52] query(): query=CREATE EXTERNAL TABLE `test_db`.`test_table2` (`a` int , `b` int , `c` int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TextFile LOCATION '/user/hive/warehouse/test_db.db/test_table2' TBLPROPERTIES("skip.header.line.count" = "1")
I1130 19:08:29.901142 18617 impala-beeswax-server.cc:426] query: Query {
01: query (string) = "CREATE EXTERNAL [...](259)",
03: configuration (list) = list<string>[0] {
},
04: hadoop_user (string) = "test_user2",
}
.
.
When location is not specified, the query is executed with ExecuteStatement() method of impala-hs2-server.cc.
But location is specified, the query is executed with query() method of impala-beeswax-server.cc.
Do you know what is wrong?
Is this a bug?
Thank you in advance.
uma66.
