Reply
Explorer
Posts: 6
Registered: ‎11-21-2016

New databases are not accessible under impala with Sentry enabled

Environment background, CDH5.9, Isilon OneFS 8, AD integrated and Sentry enabled. SSSD was configured on Cloudera cluster nodes.

 

Brief of the issue, I created one database and granted access permission to two groups. Users in two groups can see the new database in Hive. However they can’t see the database in Impala. Also hive and impala service accounts can’t see the new database.

 

Steps I have run are:

  1. Create a hiveadmin role with ALL permission on server1.
  2. Grant the hiveadmin role to AD group hiveadmin.
  3. Add hive and impala service accounts to AD group hiveadmin.
  4. Create a new database with account ryan which is also in the AD group hiveadmin.
  5. First problem, hive and impala can’t see the new database. After grant all permission on the new database to role hiveadmin, two service accounts still can’t see the database. User ryan can see the new database.
  6. Create two new roles, and grant them to same name AD groups, which are marketing-senior-analysts and marketing-junior-analysts.
  7. Grant ALL permission on the new database to the role marketing-senior-analysts.
  8. Grant SELECT permission on the new database to the role marketing-junior-analysts.
  9. Second problem, users under two AD groups can see new database in Hive, and permissions they got are also correct. Senior users can create tables but junior can’t. But users can’t see the new databases in impala.
  10. Third problem, impala service account can’t invalidate metadata. System replied no permission on the server.

Appreciated if someone can help giving some directions.

Champion
Posts: 600
Registered: ‎05-16-2016

Re: New databases are not accessible under impala with Sentry enabled

1. Does impala and Hive usess common metastore ?

2 . Please make sure your dfs.domain.socket.path in hdfs-site.xml 

3. impala-state-store , impala-catalog , impala-server - Status on the host - is it runining . 

4.  Did you restart the datanode after modifying the hdfs-site.xml 

5. do you have hive-site.xml , core-site.xml , hdfs-site.xml  under impala/conf ? 

6. can you share your impala log  - /var/log/impala 

7. Did you perform INVALIDATE METADATA in impala shell  - to get a refresh metadata 

 8 . Where you able to successful run -initSchema using schematool

 

 

 

New Contributor
Posts: 1
Registered: ‎06-27-2016

Re: New databases are not accessible under impala with Sentry enabled

To append to this issue, im getting following error after enabling sentry in the cluster.

 

05:52:15.323734  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:52:31.093621  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:52:45.323463  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:53:01.094060  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:53:15.323961  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:53:31.093381  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:53:45.323509  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:54:01.094041  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:54:15.324064  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:54:31.093245  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:54:45.323675  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:55:01.093878  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:55:15.324302  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:55:31.093950  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:55:45.323288  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:56:01.093930  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:56:15.323741  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:56:31.094040  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:56:45.324021  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:57:01.093789  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:57:15.323853  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:57:31.093240  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:57:45.324069  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:58:01.093459  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:58:15.323570  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:58:31.093924  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:58:45.323376  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:59:01.093616  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:59:15.323493  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:59:31.093808  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
05:59:45.324160  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:00:01.094159  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:00:15.323644  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:00:31.093788  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:00:45.324079  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:01:01.093439  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:01:15.323685  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:01:31.093817  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:01:45.323376  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:02:01.093240  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:02:15.323765  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:02:31.093595  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:02:45.323263  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:03:01.094116  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:03:15.323698  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:03:31.093451  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:03:45.324156  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:04:01.094097  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:04:15.323889  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:04:31.093397  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:04:45.323284  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:05:01.094367  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:05:15.324008  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:05:31.093448  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:05:45.323436  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:06:01.093889  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:06:15.324108  5707 thrift-util.cc:109] TThreadPoolServer: TServerTransport died on accept: SSL_accept: error code: 0
06:06:24.626929  5774 thrift-util.cc:109] TSocket::read() recv() <Host: ::ffff:172.18.200.31 Port: 51555>Connection reset by peer
06:06:24.626996  5774 thrift-util.cc:109] TThreadedServer client died: ECONNRESET
06:06:24.627581  5773 thrift-util.cc:109] TSocket::read() recv() <Host: ::ffff:172.18.200.31 Port: 51554>Connection reset by peer
06:06:24.628281  5773 thrift-util.cc:109] TThreadedServer client died: ECONNRESET
06:06:24.893720 16338 thrift-util.cc:109] TThreadedServer client died: SSL_read: Connection reset by peer
06:06:24.893720 12869 thrift-util.cc:109] TThreadedServer client died: SSL_read: Connection reset by peer
06:06:24.893800 16338 thrift-util.cc:109] SSL_shutdown: Broken pipe
06:06:24.893820 12869 thrift-util.cc:109] SSL_shutdown: Broken pipe
06:06:24.945636 12851 thrift-util.cc:109] TThreadedServer client died: SSL_read: Connection reset by peer
06:06:24.945636 16756 thrift-util.cc:109] TThreadedServer client died: SSL_read: Connection reset by peer
06:06:24.945736 12851 thrift-util.cc:109] SSL_shutdown: Broken pipe
06:06:24.945740 16756 thrift-util.cc:109] SSL_shutdown: Broken pipe
06:06:24.982508 12994 thrift-util.cc:109] TThreadedServer client died: SSL_read: Connection reset by peer
06:06:24.982508 12863 thrift-util.cc:109] TThreadedServer client died: SSL_read: Connection reset by peer
06:06:24.982596 12994 thrift-util.cc:109] SSL_shutdown: Broken pipe
06:06:24.982656 12863 thrift-util.cc:109] SSL_shutdown: Broken pipe

 

this error continuously poping up in the impalad daemon logs.

 

Also,

the users cannot see the complete hive databases sometimes. even if the hive session kept ideal for a while. We need to restart the connectivity again to see the complete databases.

 

Can someone help in this to identify the issue?

 

Highlighted
Champion
Posts: 600
Registered: ‎05-16-2016

Re: New databases are not accessible under impala with Sentry enabled

Announcements