Member since
10-19-2015
14
Posts
40
Kudos Received
0
Solutions
01-28-2019
07:28 PM
CVE-2018-8012 vulnerability is reported here, https://nvd.nist.gov/vuln/detail/CVE-2018-8012
o Affected Versions:
Apache ZooKeeper before 3.4.10, and 3.5.0-alpha through 3.5.3-beta.
o Mitigations:
- The Apache JIRA (ZOOKEEPER-1045) is fixed @ Zookeeper 3.4.10.
- Currently, the Fix for ZOOKEEPER-1045 (Zookeeper 3.4.10) isn't planned for any HDP release as the vulnerability is less of a vulnerability and more of a lack of a feature
- Zookeeper version 3.4.6 is present from HDP 2.3.0 to HDP 2.6.x
o Workaround:
- The workaround for this is to implement firewall rules to disallow connections from ports other than 2181.
- Customer may try to move the ZooKeeper Election and Follower ports into the privileged port range, but this would require additional logic to start ZK as root and then drop privileges which does not exist as of now.
- As such, the feasible workaround for this CVE is to configure firewall to allow communication [for Leader Election] within cluster. A possible way is:
(01) Keep the ACL of port 2181 unchanged (Any Server is allowed to access),
(02) Forbid all access to both port 2888 and 3888 as defined in the zookeeper config file zoo.cfg except for zookeeper servers (To ensure they can communicate with each other).
o Will this issue be addressed in the future versions?
As the issue has been addressed with the workaround, We have no immediate plans to backport any major features into ZooKeeper and Product Management team working on getting this added on roadmap.
This vulnerability had been reported earlier and the Jira for same had also been raised to keep a track of it. So far, the only solution for this CVE is to configure firewall to allow communication [for leader election] within cluster.
... View more
06-29-2018
12:53 PM
For an article named "in depth" this a pretty shallow article. Considering the man page for hbck is longer than this article, I guess I expected a treatise on how to interpret the output, how to determine which fix parameters to run, etc.
... View more
04-03-2017
12:47 AM
@Rohan Pednekar This is true also for any scan that requires evaluation before retrieving anything. I am not sure why this would be an HCC article. This is merely one paragraph of what could have been a well-written article about tips and tricks when dealing with HBase. I recommend looking at some of the featured articles in HCC and write that quality. This section you published could be very useful in a larger article. Thanks for your efforts.
... View more
04-01-2017
01:19 AM
6 Kudos
PROBLEM: Security scan reporting clickjacking vulnerabilities for HBase ports (16010 and 16030) and AMS ports (61330 and 61310). ROOT CAUSE:
HBASE-14148
RESOLUTION: Upgrade to the HDP 2.5.X
... View more
04-01-2017
01:11 AM
6 Kudos
PROBLEM: HBase Master failing due to following exception. 2017-03-30 20:51:13,130 ERROR [<HOST NAME>,33733,1490218864063_ChoreService_3] normalizer.RegionNormalizerChore: Caught error
java.lang.NullPointerException
at org.apache.hadoop.hbase.master.normalizer.SimpleRegionNormalizer.getRegionSize(SimpleRegionNormalizer.java:172)
at org.apache.hadoop.hbase.master.normalizer.SimpleRegionNormalizer.computePlanForTable(SimpleRegionNormalizer.java:99)
at org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:1417)
at org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:49)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ROOT CASUE: https://issues.apache.org/jira/browse/HBASE-15933 WORKAROUND: Move Splitting WAL files from hbase.root.dir (by default it set to /apps/hbase/data/WALs) to some tmp directory.
... View more
Labels:
04-01-2017
12:58 AM
5 Kudos
Append following properties in /usr/hdp/current/phoenix-client/bin/log4j.properties and change log4j.threshold=TRACE
log4j.logger.org.apache.phoenix=TRACE
log4j.logger.org.apache.hadoop.hbase.ipc=TRACE log4j.logger.org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel=DEBUG
... View more
Labels:
01-09-2018
11:37 PM
@Rohan Pednekar Issue While I apply this article, I've encountered an exception on HBase configuration:
hbase.master.loadbalancer.class=org.apache.hadoop.hbase.favored.FavoredNodeLoadBalancer
Above configuration produces following exception:
java.lang.ClassNotFoundException: Class org.apache.hadoop.hbase.favored.FavoredNodeLoadBalancer not found
Solution To fix this issue, reconfigure hbase.master.loadbalancer.class by changing the class name as followings:
hbase.master.loadbalancer.class=org.apache.hadoop.hbase.master.balancer.FavoredNodeLoadBalancer
Reference Uses of Interface org.apache.hadoop.hbase.master.LoadBalancer
... View more
02-14-2017
07:41 AM
This NPE shows that .tableinfo file does not exist. As suggested by Predrag below, run the repair tool multople times might solve the problem. (of course, sometime it might not solve the problem. We have to manually fix the issue) Another thing is that please check whether any file/directory exist for this 'prod:testj' table (via hdfs commands). I suspected that the entire table directory disappear (as I mentioned earlier, someone might make a mistake by removing this directory). In this case, another solution is to drop this table and recreate the table and re-populate data.
... View more