i built up a brand new AD environment instead of using the old one. the actual problem was it cant be resolved with group id for users. ... View more

i just noticed the error from sentry log:   2017-06-23 12:57:19,513 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: unable to return groups for user jjiang PartialGroupNameException can't execute the shell command to get the list of group id for user 'jjiang' at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:228) at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:133) at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:72) at org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:356) at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:299) at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:257) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228) at com.google.common.cache.LocalCache.get(LocalCache.java:3965) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969) at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829) at org.apache.hadoop.security.Groups.getGroups(Groups.java:215) at org.apache.sentry.provider.common.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:60) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getGroupsFromUserName(SentryPolicyStoreProcessor.java:717) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.getRequestorGroups(SentryPolicyStoreProcessor.java:684) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:552) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1017) at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:1002) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:35) at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: PartialGroupNameException Number of group names and ids do not match.    could anyone help ? ... View more

maybe not possible, as you know, it's almost done for the upgrade , and i have upgraded hdfs metadata. and i think i cant roll back without cloudera manager enterprise version. ... View more

with your command, it shows: 17/02/22 10:19:19 WARN conf.HiveConf: HiveConf of name hive.metastore.local does not exist Metastore connection URL:        jdbc:derby:;databaseName=metastore_db;create=true Metastore Connection Driver :    org.apache.derby.jdbc.EmbeddedDriver Metastore connection User:       APP Hive distribution version:       1.1.0 org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version. *** schemaTool failed ***   so we would go for hive 1.1.0, and the old version is 0.13. I'm not sure whether it matters.   and besides, i can see the error from the log where it hit:   jdbc:mysql://10.214.1.244:3306/metastore> CREATE TABLE IF NOT EXISTS `PART_CO L_STATS` ( `CS_ID` bigint(20) NOT NULL, `DB_NAME` varchar(128) CHARACTER SET lat in1 COLLATE latin1_bin NOT NULL, `TABLE_NAME` varchar(128) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL, `PARTITION_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL, `COLUMN_NAME` varchar(128) CHARACTER SET latin1 COL LATE latin1_bin NOT NULL, `COLUMN_TYPE` varchar(128) CHARACTER SET latin1 COLLAT E latin1_bin NOT NULL, `PART_ID` bigint(20) NOT NULL, `LONG_LOW_VALUE` bigint(20 ), `LONG_HIGH_VALUE` bigint(20), `DOUBLE_HIGH_VALUE` double(53,4), `DOUBLE_LOW_V ALUE` double(53,4), `BIG_DECIMAL_LOW_VALUE` varchar(4000) CHARACTER SET latin1 C OLLATE latin1_bin, `BIG_DECIMAL_HIGH_VALUE` varchar(4000) CHARACTER SET latin1 C OLLATE latin1_bin, `NUM_NULLS` bigint(20) NOT NULL, `NUM_DISTINCTS` bigint(20), `AVG_COL_LEN` double(53,4), `MAX_COL_LEN` bigint(20), `NUM_TRUES` bigint(20), `N UM_FALSES` bigint(20), `LAST_ANALYZED` bigint(20) NOT NULL, PRIMARY KEY (`CS_ID` ), CONSTRAINT `PART_COL_STATS_FK` FOREIGN KEY (`PART_ID`) REFERENCES `PARTITIONS ` (`PART_ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 0: jdbc:mysql://10.214.1.244:3306/metastore> CREATE INDEX PCS_STATS_IDX ON PART_ COL_STATS (DB_NAME,TABLE_NAME,COLUMN_NAME,PARTITION_NAME) Exit code: 1  i think the marked sql script is where it hit error.   do you have any idea about the mysql script? thanks in advance. ... View more

 i tried to change the character to latin1, it didnt work. and i tried to change the storage engine to InnoDB, it didnt work either. ... View more