Cloudera Community
sula678
user rank
Explorer

Re: RM frequent failover

by Explorer in Community Articles
‎04-12-2018 09:12 AM
‎04-12-2018 09:12 AM
Hello @kkanchu I got it, thank you very much. ... View more

Re: RM frequent failover

by Explorer in Community Articles
‎04-12-2018 01:49 AM
‎04-12-2018 01:49 AM
Hello @kkanchu Thanks for your prompt reply. Yes, the record I meant was the znode in zookeeper. When hadoop.registry.rm.enabled was enabled. I could find many empty folders under /registry/services and /registry/users If disable hadoop.registry.rm.enabled Even /registry directory was not generated. I read this document here, It said If the property hadoop.registry.rm.enabled is set to false, the RM will not interact with the registry —and the listed operations will not take place. The root paths may be created by other means, but service record cleanup will not take place. So I am curious if I disable this parameter, and don't handle this record. Will it has some unknown side effects? BTW, Slider is not used in my cluster. ... View more

Re: RM frequent failover

by Explorer in Community Articles
‎04-11-2018 12:22 PM
‎04-11-2018 12:22 PM
Hello @kkanchu I am also facing this issue. As your suggestion, disable hadoop.registry.rm.enabled But I am curious about the service record, it seems the record cleanup will not take place. In this case, where can I find the record? ... View more

Re: table row count from hive metastore

by Explorer in Support Questions
‎11-08-2017 02:20 PM
‎11-08-2017 02:20 PM
@n c It works after I set this in hive-site.xml I am not sure the reason, but it was forced to run mapreduce hive.compute.query.using.stats=false ... View more

Re: hive count(*) not working

by Explorer in Support Questions
‎11-08-2017 02:02 PM
‎11-08-2017 02:02 PM
@Sami Ahmad It works after I set this in hive-site.xml I am not sure the reason, but it was forced to run mapreduce hive.compute.query.using.stats=false I also found something in hive metastore (mine is MySQL). mysql> SELECT PARAM_VALUE FROM PARTITION_PARAMS AS A, PARTITIONS AS B WHERE A.PARAM_KEY='numRows' and A.PART_ID=B.PART_ID and A.PARAM_VALUE = 0 limit 10; It seems if PARAM_VALUE is not -1, hive will return this value without running mapreduce. If PARAM_VALUE equals to -1, mapreduce will be executed. ... View more

Re: Ranger cannot record Knox audit log to hdfs

by Explorer in Support Questions
‎07-07-2016 01:45 AM
‎07-07-2016 01:45 AM
After configured by following the document. It works now. Thank you very much. ... View more

Ranger cannot record Knox audit log to hdfs

by Explorer in Support Questions
‎07-06-2016 11:12 AM
‎07-06-2016 11:12 AM
Hello, I would like to ask a question. I integrated Ranger + Knox + LDAP on my test cluster. Ranger could records the audit logs into HDFS except the audit log of Knox. It doesn't have any user or group synchronize problem with LDAP on my cluster. Let me list up those component version of my cluster: Ambari 2.2.1 HDP 2.4.2.0 Ranger: 0.5.0.2.4 Knox: 0.6.0.2.4 The following error message is from /var/log/knox/gateway.log I cannot figure out why it couldn't recognize my cluster nameservice. My HDFS directory is hdfs://testcluster/ranger/audit 2016-07-06 19:24:18,591 ERROR queue.AuditFileSpool (AuditFileSpool.java:logError(710)) - Error sending logs to consumer. provider=knox.async.batch, consumer=knox.async.batch.hdfs 2016-07-06 19:25:18,669 ERROR provider.BaseAuditHandler (BaseAuditHandler.java:logError(329)) - Error writing to log file. java.lang.IllegalArgumentException: java.net.UnknownHostException: testcluster at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:411) at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:311) at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:678) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:150) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) at org.apache.ranger.audit.destination.HDFSAuditDestination.getLogFileStream(HDFSAuditDestination.java:221) at org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:123) at org.apache.ranger.audit.queue.AuditFileSpool.sendEvent(AuditFileSpool.java:890) at org.apache.ranger.audit.queue.AuditFileSpool.runDoAs(AuditFileSpool.java:838) at org.apache.ranger.audit.queue.AuditFileSpool$2.run(AuditFileSpool.java:759) at org.apache.ranger.audit.queue.AuditFileSpool$2.run(AuditFileSpool.java:757) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1689) at org.apache.ranger.audit.queue.AuditFileSpool.run(AuditFileSpool.java:765) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.UnknownHostException: testcluster ... 22 more Please help me figure this out. Thanks ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎03-12-2019 06:47 PM
Community Statistics
Member Since ‎07-06-2016 10:53 AM
Last Visited ‎03-12-2019 06:47 PM
Posts 7