Member since
05-22-2018
29
Posts
0
Kudos Received
0
Solutions
08-25-2019
11:59 PM
We are storing the Site-Article wise Inventory Stock and Sales in Hbase Table. With rowkey is a combination of Site+Article. There will be only one distinct record per Site-Article. However this gives a poor performance while performing Hbase Scans. If my Site=A,Article=100 my currenr rowkey design is A100. My table contains 40 million records. What can be a better rowkey design for faster scans in this usecase ?
... View more
Labels:
- Labels:
-
Apache HBase
05-31-2019
06:47 AM
I've a Hbase mapped Hive table which retrieves rows when I run select * from my_table; But when I execute select count(*) from my_table; the output is 0. What must be the issue in this case ?
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Hive
12-04-2018
07:06 AM
Hello, I added a peer through the command : add_peer 'test_replication_1',
"zk1,zk2,zk3:2181:/hbase-secure:hbase-xyzdev@XYZDEV.COM:/etc/security/keytabs/hbase.headless.keytab" Now while removing the Peer by using command : remove_peer 'test_replication_1' I'm getting the following error : ERROR : For input string: "hbase-rabfdev@RABFDEV.COM" Here is the error which I get while starting the regionserver : 2018-12-04 12:04:09,755 INFO
[regionserver/XXX.ABC3.com/XXXX:XXXX]
regionserver.HRegionServer: reportForDuty to
master=XXX.ABC2.com,16000,1543904916225 with port=16020,
startcode=15439052464362018-12-04 12:04:10,176 INFO
[regionserver/XXX.ABC3.com/XXXX:XXXX] hfile.CacheConfig:
blockCache=LruBlockCache{blockCount=0, currentSize=5396264,
freeSize=5255938776, maxSize=5261335040, heapSize=5396264, minSize=4998268416,
minFactor=0.95, multiSize=2499134208, multiFactor=0.5, singleSize=1249567104,
singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false,
cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false,
cacheDataCompressed=false, prefetchOnOpen=false2018-12-04 12:04:10,712 INFO
[regionserver/XXX.ABC3.com/XXXX:XXXX]
regionserver.HRegionServer: STOPPED: Failed initialization2018-12-04 12:04:10,713 ERROR
[regionserver/XXX.ABC3.com/XXXX:XXXX]
regionserver.HRegionServer: Failed initjava.io.IOException: Failed replication handler create at
org.apache.hadoop.hbase.replication.regionserver.Replication.initialize(Replication.java:145) at
org.apache.hadoop.hbase.regionserver.HRegionServer.newReplicationInstance(HRegionServer.java:2798) at
org.apache.hadoop.hbase.regionserver.HRegionServer.createNewReplicationInstance(HRegionServer.java:2769) at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1769) at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1513) at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:960) at
java.lang.Thread.run(Thread.java:748)Caused by:
org.apache.hadoop.hbase.replication.ReplicationException: Error adding peer
with id=test_replication_1 at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:421) at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.addExistingPeers(ReplicationPeersZKImpl.java:386) at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.init(ReplicationPeersZKImpl.java:105) at
org.apache.hadoop.hbase.replication.regionserver.Replication.initialize(Replication.java:140) ... 6 moreCaused by: java.lang.NumberFormatException: For input
string: "hbase-xyzdev@XYZDEV.COM" at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at
java.lang.Integer.parseInt(Integer.java:580) at
java.lang.Integer.parseInt(Integer.java:615) at
org.apache.hadoop.hbase.zookeeper.ZKConfig.transformClusterKey(ZKConfig.java:411) at
org.apache.hadoop.hbase.HBaseConfiguration.applyClusterKeyToConf(HBaseConfiguration.java:303) at
org.apache.hadoop.hbase.HBaseConfiguration.createClusterConf(HBaseConfiguration.java:284) at
org.apache.hadoop.hbase.HBaseConfiguration.createClusterConf(HBaseConfiguration.java:265) at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.getPeerConf(ReplicationPeersZKImpl.java:343) at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:479) at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:419) ... 9 more2018-12-04 12:04:10,716 FATAL
[regionserver/XXX.ABC3.com/XXXX:XXXX]
regionserver.HRegionServer: ABORTING region server XXX.ABC3.com,16020,1543905246436:
Unhandled: Failed replication handler createjava.io.IOException: Failed replication handler create at
org.apache.hadoop.hbase.replication.regionserver.Replication.initialize(Replication.java:145) at
org.apache.hadoop.hbase.regionserver.HRegionServer.newReplicationInstance(HRegionServer.java:2798) at
org.apache.hadoop.hbase.regionserver.HRegionServer.createNewReplicationInstance(HRegionServer.java:2769) at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1769) at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1513) at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:960) at
java.lang.Thread.run(Thread.java:748)Caused by:
org.apache.hadoop.hbase.replication.ReplicationException: Error adding peer
with id=test_replication_1 at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:421) at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.addExistingPeers(ReplicationPeersZKImpl.java:386) at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.init(ReplicationPeersZKImpl.java:105) at
org.apache.hadoop.hbase.replication.regionserver.Replication.initialize(Replication.java:140) ... 6 moreCaused by: java.lang.NumberFormatException: For input
string: "hbase-xyzdev@XYZDEV.COM" at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at
java.lang.Integer.parseInt(Integer.java:580) at
java.lang.Integer.parseInt(Integer.java:615) at
org.apache.hadoop.hbase.zookeeper.ZKConfig.transformClusterKey(ZKConfig.java:411) at
org.apache.hadoop.hbase.HBaseConfiguration.applyClusterKeyToConf(HBaseConfiguration.java:303) at
org.apache.hadoop.hbase.HBaseConfiguration.createClusterConf(HBaseConfiguration.java:284) at org.apache.hadoop.hbase.HBaseConfiguration.createClusterConf(HBaseConfiguration.java:265) at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.getPeerConf(ReplicationPeersZKImpl.java:343) at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:479) at
org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:419) ... 9 more2018-12-04 12:04:10,716 FATAL
[regionserver/XXX.ABC3.com/XXXX:XXXX]
regionserver.HRegionServer: RegionServer abort: loaded coprocessors are:
[org.apache.hadoop.hbase.security.access.AccessController]2018-12-04 12:04:11,134 INFO [main-EventThread]
zookeeper.ClientCnxn: EventThread shut down2018-12-04 12:04:11,135 INFO
[regionserver/XXX.ABC3.com/XXXX:XXXX]
regionserver.HRegionServer: stopping server
sjdchadoop03.ril.com,16020,1543905246436; zookeeper connection closed.2018-12-04 12:04:11,135 INFO
[regionserver/XXX.ABC3.com/XXXX:XXXX]
regionserver.HRegionServer: regionserver/XXX.ABC3.com/XXXX:XXXX exiting2018-12-04 12:04:11,135 ERROR [main]
regionserver.HRegionServerCommandLine: Region server exitingjava.lang.RuntimeException: HRegionServer Aborted at
org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68) at
org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87) at
org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at
org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2832)2018-12-04 12:04:11,399 INFO [pool-4-thread-1]
regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@7da10b5b2018-12-04 12:04:11,535 INFO [pool-4-thread-1]
regionserver.ShutdownHook: Starting fs shutdown hook thread.2018-12-04 12:04:11,551 INFO [pool-4-thread-1]
regionserver.ShutdownHook: Shutdown hook finished.
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Zookeeper
08-22-2018
07:15 AM
Hi @Jonathan Sneep PFA error-hbase.txt
... View more
08-22-2018
06:37 AM
Hello all, I've created a hbase integrated table in hive. I was facing the above mentioned error while creating the table, but after sometime, the table got created with no change in the query or any other parameters. But now when I'm trying to run the below query on the table, the job fails with the similar error. I've a dedicated directory under user for anirudh.deshpande. Also, I've the parameter hadoop.proxyuser.root.hosts=* What might be the issue ? Attaching exception capture.pngTrace for the same Also the Ranger configs for the same.capture1.png select count(*) from test_addresses;
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Hive
07-19-2018
04:46 AM
@sunile.manjee , We are using Hive 2.0. Can't this scenario me handled here ?
... View more
07-18-2018
10:24 AM
I'm searching for the equivalent of default syntax in Hive Hana Syntax : Create column table Employee
( NAME NVARCHAR(10),
JOIN_DATE SECONDDATE DEFAULT CURRENT_TIMESTAMP)
... View more
Labels:
- Labels:
-
Apache Hive
06-28-2018
02:13 PM
@Vinicius Higa Murakami My table is partitioned. I was able to find the size of each partitions by the steps mentioned above. However, i need to find the total size of partitions in one go. Is it feasible ?
... View more
06-28-2018
01:47 PM
I tried doing as you mentioned. But, still I'm unable to see the raw data size. It's blank. Any other method ?
... View more
06-27-2018
11:00 AM
I've created a table in Hive and loaded data in the same. Now I need to check the table size of my table. I'm using the below property to do the same: show tblproperties yourTableName("rawDataSize") However, I'm unable to know the table size. Attaching the output for the same : horton.png
... View more
Labels:
- Labels:
-
Apache Hive