Member since
01-19-2017
3676
Posts
632
Kudos Received
372
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 593 | 06-04-2025 11:36 PM | |
| 1143 | 03-23-2025 05:23 AM | |
| 572 | 03-17-2025 10:18 AM | |
| 2158 | 03-05-2025 01:34 PM | |
| 1353 | 03-03-2025 01:09 PM |
05-26-2022
12:56 AM
Thanks @Shelton for your response. I have come across this link but it is about EMR's integration. I am specifically seeking clarifications on the ranger APIs to handle the authorization of the data lying in S3. Not able to find a clear picture around this in order to begin integration with Ranger.
... View more
05-26-2022
12:07 AM
@George-Megre The Master nodes firstly are not meant for launching tasks or interaction. Edge or gateway nodes are used to run client applications and cluster administration tools. Setup of Edge/gateway node is similar to any Hadoop node except no Hadoop cluster services runs on the GW/Edge nodes they are mere entry points and connection gateway to the Master Components like HDFS (NN) HBASE etc provide you have installed the client libraries. In your case, I am sure you have the Hbase client/gateway roles on the 3 nodes and not on the master noded. The HBase client role gives the connectivity to Hbase but again I don't see why you would like to connect /initiate the HBase shell from the master node? Geoffrey
... View more
05-25-2022
11:28 PM
@TonyQiu Sorry been away for a while. Creating a table in a kerberized cluster should be possible as hbase. I just tried it out see the flow below.. I didn't need to specifically kinit as the user hbase had a valid Kerberos ticket Switch to hbase user [root@malindi keytabs]# su - hbase
Last login: Wed May 25 21:51:56 CEST 2022 Check if kerberos ticket is valid [hbase@malindi ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_1016
Default principal: hbase-jair@KENYA.KE
Valid starting Expires Service principal
05/25/2022 21:51:57 05/26/2022 21:51:57 krbtgt/KENYA.KE@KENYA.KE Connect to hbase shell as hbase [hbase@malindi ~]$ hbase-shell List existing tables hbase(main):006:0> list
TABLE
ATLAS_ENTITY_AUDIT_EVENTS
atlas_titan
2 row(s) in 0.6660 seconds
=> ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan"] Create table emp hbase(main):007:0> create 'emp', 'personal data', 'professional data'
0 row(s) in 4.9500 seconds
=> Hbase::Table - emp List tables validate emp was created hbase(main):009:0> list
TABLE
ATLAS_ENTITY_AUDIT_EVENTS
atlas_titan
emp
3 row(s) in 0.0130 seconds
=> ["ATLAS_ENTITY_AUDIT_EVENTS", "atlas_titan", "emp"] Describe table emp hbase(main):010:0> describe 'emp'
Table emp is ENABLED
emp
COLUMN FAMILIES DESCRIPTION
{NAME => 'personal data', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS
=> '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
{NAME => 'professional data', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERS
IONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}
2 row(s) in 0.8770 seconds Update table hbase(main):011:0> alter 'emp', {NAME => 'metrics', BLOCKSIZE => '16384', COMPRESSION => 'SNAPPY'}
Updating all regions with the new schema...
0/1 regions updated.
1/1 regions updated.
Done.
0 row(s) in 4.9660 seconds Can you share your snippet?
... View more
05-19-2022
02:47 PM
i got this error when enable the hbase backup as below on hbase-site.xml <property> <name>hbase.backup.enable</name> <value>true</value> </property> <property> <name>hbase.master.logcleaner.plugins</name> <value>org.apache.hadoop.hbase.backup.master.BackupLogCleaner,...</value> </property> <property> <name>hbase.procedure.master.classes</name> <value>org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager,...</value> </property> <property> <name>hbase.procedure.regionserver.classes</name> <value>org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager,...</value> </property> <property> <name>hbase.coprocessor.region.classes</name> <value>org.apache.hadoop.hbase.backup.BackupObserver,...</value> </property> <property> <name>hbase.master.hfilecleaner.plugins</name> <value>org.apache.hadoop.hbase.backup.BackupHFileCleaner,...</value> </property> <property> <name>hbase.cluster.distributed</name> <value>false</value> </property> <property> <name>hbase.tmp.dir</name> <value>./tmp</value> </property> <property> <name>hbase.unsafe.stream.capability.enforce</name> <value>false</value> </property> </configuration>
... View more
04-05-2022
07:43 AM
@araujo do you have any suggestion for this case ?
... View more
03-29-2022
12:17 PM
1 Kudo
Hi @Shelton, sorry for the delay in responding. I finally found the solution to my problem. I need to specify the column properties and then Compute Stats will work perfectly. ```sql ALTER TABLE myschema.mytable ADD COLUMNS (mycolumn1 TIMESTAMP NULL COMPRESSION DEFAULT_COMPRESSION, mycolumn2 TIMESTAMP NULL COMPRESSION DEFAULT_COMPRESSION); ```
... View more
02-02-2022
10:03 PM
@Shelton As you have mentioned solution in Point 3, 3. There seems to be a problem with hiveserver creating a znode in zookeeper. [caught exception: ZooKeeper node /hiveserver2 is not ready yet] How can I create hiveserver2 znode instance in zookeeper if its not created?.
... View more
01-16-2022
09:53 AM
@Koffi This is typical of a rogue process hasn't reslease the Caused by: java.net.BindException: Address already in use You will need to run # kill -9 5356 The restart the NN that should resolve the issue
... View more
01-11-2022
09:44 AM
Hi, even I am facing the same trouble when trying to load Ambari web page, it shows 502 error. It was working fine 2 days back but the error is showing from today when I tried to login in the morning. I have tried all the replies stated above, none worked. I will be grateful if someone can resolve this issue at the earliest. Many Thanks
... View more
01-05-2022
08:27 PM
If we change the replication factor from Cloudera or Ambari console will it work for existing blocks as well?
... View more