Support Questions

zimo80 · ‎11-04-2016

Hi all,

- CDH 5.7.1
- Hive 1.1.0
- Impala 2.5.0
- Sentry service Enable
- NO LDAP
- NO Kerberos
I have installed and configured Sentry Service from CM.
In HUE i have created user "hive"and relative group ("hive") with all privilege (hue privilege)
When i try to create a new role i receveid thi error:

hive         ERROR    could not create role
Traceback (most recent call last):
  File "/opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hue/apps/security/src/security/api/hive.py", line 176, in create_role
    api.create_sentry_role(role['name'])
  File "/opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hue/desktop/libs/libsentry/src/libsentry/api.py", line 49, in decorator
    raise e
SentryException: Unknown error for request: TCreateSentryRoleRequest(protocol_version:2, requestorUserName:hive, roleName:prova), message: Error(s) were found while auto-creating/validating the datastore for classes. The errors are printed in the log, and are attached to this exception.

I try to create role from beeline and receveid the same error

command:

0: jdbc:hive2://bda1node04:10000> create role role1;

error:

 
2016-11-03 14:43:32,563 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-55]: </PERFLOG method=Driver.execute start=1478180612438 end=1478180612563 duration=125 from=org.apache.hadoop.hive.ql.Driver>
2016-11-03 14:43:32,563 INFO  org.apache.hadoop.hive.ql.Driver: [HiveServer2-Background-Pool: Thread-55]: Completed executing command(queryId=hive_20161103144343_efb717ff-b63c-408c-90f6-a93944eede95); Time taken: 0.125 seconds
2016-11-03 14:43:32,563 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-55]: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2016-11-03 14:43:32,563 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-55]: </PERFLOG method=releaseLocks start=1478180612563 end=1478180612563 duration=0 from=org.apache.hadoop.hive.ql.Driver>
2016-11-03 14:43:32,564 ERROR org.apache.hive.service.cli.operation.Operation: [HiveServer2-Background-Pool: Thread-55]: Error running hive query:
org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. Unknown error for request: TCreateSentryRoleRequest(protocol_version:2, requestorUserName:hive, roleName:role1), message: Error(s) were found while auto-creating/validating the datastore for classes. The errors are printed in the log, and are attached to this exception.. Server Stacktrace: javax.jdo.JDODataStoreException: Error(s) were found while auto-creating/validating the datastore for classes. The errors are printed in the log, and are attached to this exception.
        at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)
        at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)
        at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)
        at org.apache.sentry.provider.db.service.persistent.SentryStore.createSentryRole(SentryStore.java:316)
        at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.create_sentry_role(SentryPolicyStoreProcessor.java:230)
        at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$create_sentry_role.getResult(SentryPolicyService.java:893)
        at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$create_sentry_role.getResult(SentryPolicyService.java:878)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35)
        at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
NestedThrowablesStackTrace:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 3072 bytes
        at sun.reflect.GeneratedConstructorAccessor21.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
        at com.mysql.jdbc.Util.getInstance(Util.java:360)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:978)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2484)
        at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:848)
        at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:742)
        at com.jolbox.bonecp.StatementHandle.execute(StatementHandle.java:300)
        at org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatement(AbstractTable.java:760)
        at org.datanucleus.store.rdbms.table.TableImpl.createIndices(TableImpl.java:648)
        at org.datanucleus.store.rdbms.table.TableImpl.createConstraints(TableImpl.java:422)
        at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:3487)
        at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.addClassTablesAndValidate(RDBMSStoreManager.java:3218)
        at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2869)
        at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:122)
        at org.datanucleus.store.rdbms.RDBMSStoreManager.addClasses(RDBMSStoreManager.java:1606)
        at org.datanucleus.store.AbstractStoreManager.addClass(AbstractStoreManager.java:954)
        at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:680)

Have you any suggestions?

thanks

zimo80 · ‎11-04-2016

I finaly solved the problem!

probably was a problem of creating tables on mysql.

to solve this error:

Error: Index column size too large. The maximum column size is 767 bytes

It's necessary to create the sentry tables using ROW_FORMAT = DYNAMIC.

In the cloudera version CDH 5.7.1.1 in the scritps /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/sentry/scripts/sentrystore/upgrade/sentry-mysql-1.5.0.sql

the ROW_FORMAT option is omitted, the default is COMPACT.

I must use the scripts of version CDH 5.7.0.1 /opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p1464.1349/lib/sentry/scripts/sentrystore/upgrade/sentry-mysql-1.5.0.sql

where the option ROW_FORMAT = DYNAMIC is explicit.

With this script i created correctly the sentry table.

After I restart the service i have created the role on HUE and beeline without any problem.

My only doubt is, why cdh 5.7.1.1 doesn't contain the option ROW_FORMAT?

View solution in original post

zimo80 · ‎11-04-2016

I try to recreate Sentry Database Table from CM(

https://cloudera-portal.force.com/articles/KB_Article/Recreating-Database-Tables-Sentry-Service?popu...

)

, i received this error:

+ exec /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop/bin/hadoop jar /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/sentry/lib/sentry-core-common-1.5.1-cdh5.7.1.jar
 org.apache.sentry.SentryMain --command schema-tool --verbose --conffile /var/run/cloudera-scm-agent/process/3000-sentry-create-dbtables/sentry-site.xml --dbType mysql --initSchema 
Connecting to jdbc:mysql://bda1node03.sede.corp.sanpaoloimi.com:3306/sentry?useUnicode=true&characterEncoding=UTF-8 
Connected to: MySQL (version 5.6.30-enterprise-commercial-advanced-log) 
Driver: MySQL Connector Java (version mysql-connector-java-commercial-5.1.34 ( Revision: jess.balint@oracle.com-20141014163213-wqbwpf1ok2kvo1om )) 
Transaction isolation: TRANSACTION_REPEATABLE_READ 
Error: Index column size too large. The maximum column size is 767 bytes. (state=HY000,code=1709) 
org.apache.sentry.SentryUserException: Schema initialization FAILED! Metastore state would be inconsistent !! 
org.apache.sentry.SentryUserException: Schema initialization FAILED! Metastore state would be inconsistent !! 
	at org.apache.sentry.provider.db.tools.SentrySchemaTool.doInit(SentrySchemaTool.java:322) 
	at org.apache.sentry.provider.db.tools.SentrySchemaTool.doInit(SentrySchemaTool.java:295) 
	at org.apache.sentry.provider.db.tools.SentrySchemaTool$CommandImpl.run(SentrySchemaTool.java:569) 
	at org.apache.sentry.SentryMain.main(SentryMain.java:114) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
	at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) 
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 
Caused by: java.io.IOException: Schema script failed, errorcode 2 
	at org.apache.sentry.provider.db.tools.SentrySchemaTool.runBeeLine(SentrySchemaTool.java:423) 
	at org.apache.sentry.provider.db.tools.SentrySchemaTool.runBeeLine(SentrySchemaTool.java:396) 
	at org.apache.sentry.provider.db.tools.SentrySchemaTool.doInit(SentrySchemaTool.java:318) 
... 9 more *** Sentry schemaTool failed ***

The InnoDB storage engine is supported

zimo80 · ‎11-04-2016