Re: Understanding HBase HDFS usage

tyu · ‎01-03-2017

I would say it is not recommended (I am the person who ported MOB to HDP) There hasn't been any testing on using 2.5 jars on top of 2.4 Is there particular reason why the customer cannot upgrade to 2.5 ? Thanks

tyu · ‎12-20-2016

servlet-api-2.5.jar is brought in by transitive dependency: [INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3.2.6.0.0-SNAPSHOT:compile [INFO] | +- org.apache.hadoop:hadoop-yarn-common:jar:2.7.3.2.6.0.0-SNAPSHOT:compile [INFO] | | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile [INFO] | | | \- javax.activation:activation:jar:1.1:compile [INFO] | | +- javax.servlet:servlet-api:jar:2.5:compile

tyu · ‎12-20-2016

netty-3.2.4.Final.jar is brought in by hbase-it module. It shouldn't collide with netty-all-4.0.23.Final.jar

tyu · ‎12-15-2016

HDP 2.5 has been released. You can use the MOB feature now.

tyu · ‎09-17-2016

You can take a look at: http://hbase.apache.org/book.html#ops.snapshots Especially: http://hbase.apache.org/book.html#ops.snapshots.export The backup / restore feature in HDP 2.5 makes backing up multiple tables easy to operate.

tyu · ‎09-06-2016

How many regions are there for this table ? What's the value for hbase.hregion.max.filesize ? Thanks

tyu · ‎09-06-2016

Can you attach region server log for the server which hosts the single large region ? There should be some clue in the region server log.

tyu · ‎09-06-2016

For HDP 2.3 (Apache 1.1.2), ./hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseConfiguration.java calls HeapMemorySizeUtil.checkForClusterFreeMemoryLimit(conf); There is no HBaseConfiguration.checkForClusterFreeMemoryLimit Can you double check your classpath to see which hbase related jars are present. Please pastebin those jars Thanks

tyu · ‎09-06-2016

Which version of hbase is used by nutch ? Thanks

tyu · ‎09-06-2016

Please check the value of hbase.regionserver.global.memstore.upperLimit

tyu · ‎09-03-2016

Which user did you use to run the code ? What's the output of the following command ? klist -kt /etc/security/keytabs/hbase.service.keytab Normally hbase.service.keytab should be used by user 'hbase'. Please illustrate your use case in more detail. Please take a look at hbase-common/src/main/java/org/apache/hadoop/hbase/AuthUtil.java

tyu · ‎09-02-2016

Pig doesn't support creating hbase table. Please pre-create table in hbase.

tyu · ‎09-01-2016

We need more of the master log beyond what tail showed us. Please attach master log.

tyu · ‎09-01-2016

Can you attach the master log so that we can better help you ? You can add hbase.master.namespace.init.timeout to hbase-site.xml by using Ambari. Still, finding the root cause is desirable.

tyu · ‎09-01-2016

@dengke li Can you attach master log ? You should find it under /grid/0/log/hbase/ Consider increasing hbase.master.namespace.init.timeout Default is 300000 ms

tyu · ‎08-30-2016

NoRouteToHostException occurred in both region server and node manager logs. Please check network connectivity.

tyu · ‎08-30-2016

w.r.t. AWS instance check failure, have you looked at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstances.html

tyu · ‎08-29-2016

Zack: Can you check other regions which failed to open (such as a97029c18889b3b3168d11f910ef04ae ) ?

tyu · ‎08-29-2016

Zack: You can use hfile tool to inspect: MY_BROKEN_TABLE/8a444fa1979524e97eb002ce8aa2d7aa/0/4f9a5c26ddb0413aa4eb64a869ab4a2c http://hbase.apache.org/book.html#hfile_tool

tyu · ‎08-29-2016

'hbase.master.logcleaner.plugins' is not overridden in hbase-site.xml Need to investigate further.

tyu · ‎08-26-2016

Snippet from master log: http://pastebin.com/4iv5YKEQ Here is related code: String plugins = conf.get(HBASE_MASTER_LOGCLEANER_PLUGINS); Looks like the value for "hbase.master.logcleaner.plugins" was null - default should be org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner Can you attach hbase-site.xml ? Thanks

tyu · ‎08-22-2016

I have sent an email to Roshan who is more familiar with Flume component.

tyu · ‎08-22-2016

Here is the code in HBaseSink.java: if (r instanceof Put) { ((Put) r).setWriteToWAL(enableWal); Here is code in Put.java of HDP 2.3: public Put setWriteToWAL(boolean write) { return (Put) super.setWriteToWAL(write); In pom.xml of Flume in HDP 2.3: <hbaseversion>0.94.2</hbaseversion> Planning to raise an internal JIRA for this incompatibility.

tyu · ‎08-22-2016

For #2, this is currently not possible. You need to upgrade the whole stack.

tyu · ‎08-18-2016

The installation is controlled by the rpm packages. It is hard coded to /usr/hdp

tyu · ‎08-17-2016

Can you provide more information (attaching region server log) ? Load balancer wouldn't create new region.

tyu · ‎08-01-2016

Which release of HDP are you using ?

tyu · ‎08-01-2016

One aspect to note w.r.t. using PrefixFilter is that the start row is not automatically set. You need to pass the correct start row along with PrefixFilter. Otherwise the number of rows scanned may be quite high.

tyu · ‎07-29-2016

@Ashnee: You can also use snapshot for table backup: http://hbase.apache.org/book.html#ops.snapshots

tyu · ‎07-21-2016

ACLs were not part of the core hbase (implemented through coprocessor). We were adding to a core functionality knowledge about an external component (ACL). There was the discussion about the meaning of restoring the acls that we snapshotted (see HBASE-11013). Please consult with the following for up-to-date ACL: http://hbase.apache.org/book.html#appendix_acl_matrix We cannot allow any user to restore any snapshot otherwise you'll be able to see data that is not yours. There is no ACL on snapshot to say "allow this user to restore/clone" the snapshot.

Online	Offline
Last Visited	‎09-14-2018 10:46 PM

Date Registered	‎10-22-2015 03:57 PM
Last Visited	‎09-14-2018 10:46 PM
Total Messages Posted	241
Total Kudos Received	85

Re: Understanding HBase HDFS usage

Re: Hbase MOB and HDP2.4

Re: HBase jars with conflicting versions?

Re: HBase client failing to connect to Kerberized ...

Re: Can we create HBase table using PIG if Yes the...

Re: Hbase MOB and HDP2.4

Re: HBase jars with conflicting versions?

Re: HBase jars with conflicting versions?

Re: Hbase Storing pdf and Retrieval

Re: hbase table copy from one cluster to other

Re: HBase BulkLoad - Region Split behaviour

Re: HBase BulkLoad - Region Split behaviour

Re: failed to start nutch with hbase

Re: failed to start nutch with hbase

Re: failed to start nutch with hbase

Re: HBase client failing to connect to Kerberized ...

Re: Can we create HBase table using PIG if Yes the...

Re: org.apache.hadoop.hbase.PleaseHoldException: M...

Re: org.apache.hadoop.hbase.PleaseHoldException: M...

Re: org.apache.hadoop.hbase.PleaseHoldException: M...

Re: How to debug cluster network issues?

Re: How to debug cluster network issues?

Re: How to fix "offline regions" in HBase

Re: How to fix "offline regions" in HBase

Re: Hbase Master and Reg Server shutting down on H...

Re: Hbase Master and Reg Server shutting down on H...

Re: HDP 2.3.2 and HBase 0.98 Compatibility

Re: HDP 2.3.2 and HBase 0.98 Compatibility

Re: HDP 2.3.2 and HBase 0.98 Compatibility

Re: can we customize HDP installation in other dir...

Re: Hbase region split

Re: Testing HBase bucket cache

Re: HBase Java API scan is too slow

Re: Hbase backup

Re: Why is global admin permissions necessary to c...