We are storing the Site-Article wise Inventory Stock and Sales in Hbase Table. With rowkey is a combination of Site+Article. There will be only one distinct record per Site-Article. However this gives a poor performance while performing Hbase Scans. If my Site=A,Article=100 my currenr rowkey design is A100. My table contains 40 million records. What can be a better rowkey design for faster scans in this usecase ? ... View more

Hello, I added a peer through the command : add_peer 'test_replication_1', "zk1,zk2,zk3:2181:/hbase-secure:hbase-xyzdev@XYZDEV.COM:/etc/security/keytabs/hbase.headless.keytab" Now while removing the Peer by using command : remove_peer 'test_replication_1' I'm getting the following error : ERROR : For input string: "hbase-rabfdev@RABFDEV.COM" Here is the error which I get while starting the regionserver : 2018-12-04 12:04:09,755 INFO [regionserver/XXX.ABC3.com/XXXX:XXXX] regionserver.HRegionServer: reportForDuty to master=XXX.ABC2.com,16000,1543904916225 with port=16020, startcode=15439052464362018-12-04 12:04:10,176 INFO [regionserver/XXX.ABC3.com/XXXX:XXXX] hfile.CacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=5396264, freeSize=5255938776, maxSize=5261335040, heapSize=5396264, minSize=4998268416, minFactor=0.95, multiSize=2499134208, multiFactor=0.5, singleSize=1249567104, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false2018-12-04 12:04:10,712 INFO [regionserver/XXX.ABC3.com/XXXX:XXXX] regionserver.HRegionServer: STOPPED: Failed initialization2018-12-04 12:04:10,713 ERROR [regionserver/XXX.ABC3.com/XXXX:XXXX] regionserver.HRegionServer: Failed initjava.io.IOException: Failed replication handler create at org.apache.hadoop.hbase.replication.regionserver.Replication.initialize(Replication.java:145) at org.apache.hadoop.hbase.regionserver.HRegionServer.newReplicationInstance(HRegionServer.java:2798) at org.apache.hadoop.hbase.regionserver.HRegionServer.createNewReplicationInstance(HRegionServer.java:2769) at org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1769) at org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1513) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:960) at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.hadoop.hbase.replication.ReplicationException: Error adding peer with id=test_replication_1 at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:421) at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.addExistingPeers(ReplicationPeersZKImpl.java:386) at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.init(ReplicationPeersZKImpl.java:105) at org.apache.hadoop.hbase.replication.regionserver.Replication.initialize(Replication.java:140) ... 6 moreCaused by: java.lang.NumberFormatException: For input string: "hbase-xyzdev@XYZDEV.COM" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.parseInt(Integer.java:615) at org.apache.hadoop.hbase.zookeeper.ZKConfig.transformClusterKey(ZKConfig.java:411) at org.apache.hadoop.hbase.HBaseConfiguration.applyClusterKeyToConf(HBaseConfiguration.java:303) at org.apache.hadoop.hbase.HBaseConfiguration.createClusterConf(HBaseConfiguration.java:284) at org.apache.hadoop.hbase.HBaseConfiguration.createClusterConf(HBaseConfiguration.java:265) at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.getPeerConf(ReplicationPeersZKImpl.java:343) at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:479) at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:419) ... 9 more2018-12-04 12:04:10,716 FATAL [regionserver/XXX.ABC3.com/XXXX:XXXX] regionserver.HRegionServer: ABORTING region server XXX.ABC3.com,16020,1543905246436: Unhandled: Failed replication handler createjava.io.IOException: Failed replication handler create at org.apache.hadoop.hbase.replication.regionserver.Replication.initialize(Replication.java:145) at org.apache.hadoop.hbase.regionserver.HRegionServer.newReplicationInstance(HRegionServer.java:2798) at org.apache.hadoop.hbase.regionserver.HRegionServer.createNewReplicationInstance(HRegionServer.java:2769) at org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1769) at org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1513) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:960) at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.hadoop.hbase.replication.ReplicationException: Error adding peer with id=test_replication_1 at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:421) at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.addExistingPeers(ReplicationPeersZKImpl.java:386) at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.init(ReplicationPeersZKImpl.java:105) at org.apache.hadoop.hbase.replication.regionserver.Replication.initialize(Replication.java:140) ... 6 moreCaused by: java.lang.NumberFormatException: For input string: "hbase-xyzdev@XYZDEV.COM" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.parseInt(Integer.java:615) at org.apache.hadoop.hbase.zookeeper.ZKConfig.transformClusterKey(ZKConfig.java:411) at org.apache.hadoop.hbase.HBaseConfiguration.applyClusterKeyToConf(HBaseConfiguration.java:303) at org.apache.hadoop.hbase.HBaseConfiguration.createClusterConf(HBaseConfiguration.java:284) at org.apache.hadoop.hbase.HBaseConfiguration.createClusterConf(HBaseConfiguration.java:265) at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.getPeerConf(ReplicationPeersZKImpl.java:343) at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:479) at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:419) ... 9 more2018-12-04 12:04:10,716 FATAL [regionserver/XXX.ABC3.com/XXXX:XXXX] regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: [org.apache.hadoop.hbase.security.access.AccessController]2018-12-04 12:04:11,134 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down2018-12-04 12:04:11,135 INFO [regionserver/XXX.ABC3.com/XXXX:XXXX] regionserver.HRegionServer: stopping server sjdchadoop03.ril.com,16020,1543905246436; zookeeper connection closed.2018-12-04 12:04:11,135 INFO [regionserver/XXX.ABC3.com/XXXX:XXXX] regionserver.HRegionServer: regionserver/XXX.ABC3.com/XXXX:XXXX exiting2018-12-04 12:04:11,135 ERROR [main] regionserver.HRegionServerCommandLine: Region server exitingjava.lang.RuntimeException: HRegionServer Aborted at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:68) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2832)2018-12-04 12:04:11,399 INFO [pool-4-thread-1] regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@7da10b5b2018-12-04 12:04:11,535 INFO [pool-4-thread-1] regionserver.ShutdownHook: Starting fs shutdown hook thread.2018-12-04 12:04:11,551 INFO [pool-4-thread-1] regionserver.ShutdownHook: Shutdown hook finished. ... View more

Hello all, I've created a hbase integrated table in hive. I was facing the above mentioned error while creating the table, but after sometime, the table got created with no change in the query or any other parameters. But now when I'm trying to run the below query on the table, the job fails with the similar error. I've a dedicated directory under user for anirudh.deshpande. Also, I've the parameter hadoop.proxyuser.root.hosts=* What might be the issue ? Attaching exception capture.pngTrace for the same Also the Ranger configs for the same.capture1.png select count(*) from test_addresses; ... View more

I'm searching for the equivalent of default syntax in Hive Hana Syntax : Create column table Employee ( NAME NVARCHAR(10), JOIN_DATE SECONDDATE DEFAULT CURRENT_TIMESTAMP) ... View more

I've created a table in Hive and loaded data in the same. Now I need to check the table size of my table. I'm using the below property to do the same: show tblproperties yourTableName("rawDataSize") However, I'm unable to know the table size. Attaching the output for the same : horton.png ... View more