Member since
03-27-2014
27
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
6075 | 06-27-2017 01:33 AM |
12-25-2017
12:22 AM
1 Kudo
i built up a brand new AD environment instead of using the old one. the actual problem was it cant be resolved with group id for users.
... View more
06-27-2017
01:33 AM
this is sloved with building up with a new AD. the problem was some goup id cant be resolved.
... View more
06-24-2017
11:12 PM
i just noticed the error from sentry log: 2017-06-23 12:57:19,513 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: unable to return groups for user jjiang PartialGroupNameException can't execute the shell command to get the list of group id for user 'jjiang' at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:228) at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:133) at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:72) at org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:356) at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:299) at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:257) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at com.google.common.cache.LocalCache.get(LocalCache.java:3965) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) at org.apache.hadoop.security.Groups.getGroups(Groups.java:215) at org.apache.sentry.provider.common.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:60) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getGroupsFromUserName(SentryPolicyStoreProcessor.java:717) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getRequestorGroups(SentryPolicyStoreProcessor.java:684) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:552) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1017) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1002) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35) at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: PartialGroupNameException Number of group names and ids do not match. could anyone help ?
... View more
06-24-2017
02:18 AM
hi, i actually met an issue when configure sentry with hive. basically if i configure hue security with desktop.auth.backend.AllowFirstUserDjangoBackend, and use a local user to query hive, it would be succesful to query the right database. but if i configure hue with AD server, and use an AD user to logon hue, it would not load any database that the user was supposed to have privileges on. the AD user looks have more gourps: [root@ ~]# id jjiang uid=16777217(jjiang) gid=16777216(domain users) groups=16777216(domain users),502(db_admin),16777225,16777217(apacsysadmin),16777218(apac),16777226,16777239(apac_connectivity),16777227,16777219(itsystems),16777220(apac_it),16777228,16777247(apac_marketing_services),16777229,16777221(nantong office),16777230,16777222(ap_all_okta_users),16777223(okta_o365_users),16777224(ap-fp-general-users),16777231(ap_slack_itops) is it a problem with so many groups? the actual group i granted permission is db_admin.
... View more
Labels:
- Labels:
-
Apache Sentry
02-22-2017
06:13 PM
maybe not possible, as you know, it's almost done for the upgrade , and i have upgraded hdfs metadata. and i think i cant roll back without cloudera manager enterprise version.
... View more
02-21-2017
06:33 PM
with your command, it shows: 17/02/22 10:19:19 WARN conf.HiveConf: HiveConf of name hive.metastore.local does not exist Metastore connection URL: jdbc:derby:;databaseName=metastore_db;create=true Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver Metastore connection User: APP Hive distribution version: 1.1.0 org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version. *** schemaTool failed *** so we would go for hive 1.1.0, and the old version is 0.13. I'm not sure whether it matters. and besides, i can see the error from the log where it hit: jdbc:mysql://10.214.1.244:3306/metastore> CREATE TABLE IF NOT EXISTS `PART_CO
L_STATS` ( `CS_ID` bigint(20) NOT NULL, `DB_NAME` varchar(128) CHARACTER SET lat
in1 COLLATE latin1_bin NOT NULL, `TABLE_NAME` varchar(128) CHARACTER SET latin1
COLLATE latin1_bin NOT NULL, `PARTITION_NAME` varchar(767) CHARACTER SET latin1
COLLATE latin1_bin NOT NULL, `COLUMN_NAME` varchar(128) CHARACTER SET latin1 COL
LATE latin1_bin NOT NULL, `COLUMN_TYPE` varchar(128) CHARACTER SET latin1 COLLAT
E latin1_bin NOT NULL, `PART_ID` bigint(20) NOT NULL, `LONG_LOW_VALUE` bigint(20
), `LONG_HIGH_VALUE` bigint(20), `DOUBLE_HIGH_VALUE` double(53,4), `DOUBLE_LOW_V
ALUE` double(53,4), `BIG_DECIMAL_LOW_VALUE` varchar(4000) CHARACTER SET latin1 C
OLLATE latin1_bin, `BIG_DECIMAL_HIGH_VALUE` varchar(4000) CHARACTER SET latin1 C
OLLATE latin1_bin, `NUM_NULLS` bigint(20) NOT NULL, `NUM_DISTINCTS` bigint(20),
`AVG_COL_LEN` double(53,4), `MAX_COL_LEN` bigint(20), `NUM_TRUES` bigint(20), `N
UM_FALSES` bigint(20), `LAST_ANALYZED` bigint(20) NOT NULL, PRIMARY KEY (`CS_ID`
), CONSTRAINT `PART_COL_STATS_FK` FOREIGN KEY (`PART_ID`) REFERENCES `PARTITIONS
` (`PART_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1
0: jdbc:mysql://10.214.1.244:3306/metastore> CREATE INDEX PCS_STATS_IDX ON PART_
COL_STATS (DB_NAME,TABLE_NAME,COLUMN_NAME,PARTITION_NAME) Exit code: 1 i think the marked sql script is where it hit error. do you have any idea about the mysql script? thanks in advance.
... View more
02-21-2017
01:24 AM
i tried to change the character to latin1, it didnt work. and i tried to change the storage engine to InnoDB, it didnt work either.
... View more
02-20-2017
07:32 PM
hi did a CDH upgrade from CDH5.2 to CDH5.10 with cloudera manager. and it failed in the step of upgrade hive schema, shows the error as below: Error: Specified key was too long; max key length is 1000 bytes (state=42000,code=1071)
Closing: 0: jdbc:mysql://*:3306/metastore?useUnicode=true&characterEncoding=UTF-8
org.apache.hadoop.hive.metastore.HiveMetaException: Upgrade FAILED! Metastore state would be inconsistent !!
org.apache.hadoop.hive.metastore.HiveMetaException: Upgrade FAILED! Metastore state would be inconsistent !!
at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:229)
at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:197)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:468)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: Schema script failed, errorcode 2
at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:358)
at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:326)
at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:224)
... 8 more
*** schemaTool failed *** could anyone help me?
... View more
Labels:
- Labels:
-
Apache Hive