About sheshk11

sheshk11 · ‎06-08-2021

@GangWar Thank you!

sheshk11 · ‎06-06-2021

Hi All, We have Hue (version 4.2.0) in our prod environment and it is coupled with our LDAP. We manually onboard new users by going to "Manage Users" -> "Add/Sync Manage Users". Here, we add the LDAP ID and later when the user gets added. Once the user is added, we again search the newly added LDAP user, click on that and go to "Profiles and Group". Here, we select "temporary-access" so that User can able to access the UI. Without Temporary-access selection, user will not be able to see the UI. We want to get this automated by writing a Py program and we are looking for the right REST API for adding users to Hue and give him/her "temp-access" in profiles and group. The following link wasn't helpful - http://cloudera.github.io/hue/latest/developer/api/ Please assist us. I would highly appreciate any form of assistance or help. Regards, Shesh Kumar

sheshk11 · ‎05-27-2019

@Geoffrey Shelton Okot : Thank you for the detailed explanation.

sheshk11 · ‎05-27-2019

@Geoffrey Shelton Okot I do not see that option 😞

sheshk11 · ‎05-21-2019

Hi, As per my knowledge (please correct me if I am wrong), the Datanodes sends the block report to both Active and Standby Namenodes. The job of Active NN is to write to the Journal Nodes and the job of Standby namenode is to read from Journal nodes. Now why does Standby namenode need to read from Journal nodes when the Datanodes (slaves) are already sending the block reports to it?

sheshk11 · ‎05-08-2019

My cluster was hung. Was unable to add hosts or perform any basic activities in Ambari like restart of a service. Was constantly seeing the WARN snippet in Ambari Server logs: Unable to lookup the cluster by ID; assuming that there is no cluster and therefore no configs for this execution command: Cluster not found, clusterName=clusterID=-1 Here's a small hack to resolve the issue: 1. Check the cluster id in your backend Ambari DB. Mine is MySQL. select * from clusterstate; 2. The same value found in step 1 should be there in Stage table's "cluster_id" columns select stage_id, request_id, cluster_id from stage; 3. If there are values as -1 please update it to the correct value found in step 1. Example: UPDATE stage SET cluster_id='2' WHERE request_id IN (383,384,388,389); 4. Restart Ambari-Server ambari-server restart 5. Post this check by restarting any service like Grafana or any small service which not does impact the Hadoop service. If it proceeds, the cluster is now stable and you will be able to add nodes. 6. If issue persists, the perform the following in your backend Ambari DB. SELECT * FROM host_role_command WHERE status='PENDING'; 7. If you get any output, you need to update the status to "ABORTED". UPDATE host_role_command SET status='ABORTED' WHERE status='PENDING'; 8. Restart Ambari-Server ambari-server restart Validate the health of Ambari by restarting Grafana or any small service which not does impact the Hadoop service. If everything is good, proceed by adding the nodes.

sheshk11 · ‎03-02-2019

Hi, No need to do any deletion. Just follow the steps here to resolve the issue without any data loss: https://community.hortonworks.com/questions/242343/hbase-table-is-stuck-in-disabling-state-neither-en.html Thanks, Shesh

sheshk11 · ‎03-02-2019

This is just a knowledge sharing article. I had faced this issue in production and took me a day to resolve it. The workaround that I'm sharing will help you in getting your table back online in "Enabled" without deleting Zookeeper Hbase table znode or any data as a matter of fact.. Here are steps to resolve it. 1. Run a "get" command against hbase:meta for the affected table hbase(main):003:0> get 'hbase:meta', '<AFFECTED_TABLE_NAME>', 'table:state' COLUMN CELL table:state timestamp=1551456805377, value=\x08\x02 2. Notice the above "value". Its pointing to \x08\x02 which is wrong. The value should either be \x08\x00 (Enabled) or \x08\x01 (Disabled) 3. Edit the value manually. hbase(main):003:0> put 'hbase:meta','<AFFECTED_TABLE_NAME>','table:state',"\b\0" Click here for more information on "Control Characters" 4. Verify the same. hbase(main):003:0> get 'hbase:meta', '<AFFECTED_TABLE_NAME>', 'table:state' The "value" now should be \x08\x00 Post this, again run disable <table_name> and enable <table_name> just for the love of sanity check in hbase shell and you are done with the issue.

sheshk11 · ‎12-18-2018

I'm getting the following error after following "Option #2". My Ranger is not TLS/SSL enabled. Please help 18 Dec 2018 12:17:23 INFO LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - Using principal = rangerusersync/stg-agent001-stg-cloud009.XXXXX.nm2@XXXXXX.COM and keytab = /etc/security/keytabs/rangerusersync.service.keytab 18 Dec 2018 12:17:24 ERROR LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - Failed to build Group List : com.sun.jersey.api.client.UniformInterfaceException: POST http://stg-agent001-stg-cloud009.xxxxx.nm2:6080/service/xusers/groups/ returned a response status of 404 Not Found and 18 Dec 2018 12:17:24 INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - LdapDeltaUserGroupBuilder.getUsers() completed with user count: 0 18 Dec 2018 12:17:24 ERROR LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - Failed to add User : com.sun.jersey.api.client.UniformInterfaceException: POST http://stg-agent001-stg-cloud009.xxxxx.nm2:6080/service/xusers/ugsync/auditinfo/ returned a response status of 404 Not Found

sheshk11 · ‎12-12-2018

Hi, I'm seeing this error in RS logs. Can someone help me resolve the issue? 2018-12-12 18:28:54,407 INFO [B.defaultRpcServer.handler=12,queue=0,port=16020] shortcircuit.ShortCircuitCache: ShortCircuitCache(0x3d630f8a): could not load 1093156401_BP-853897652-10.84.192.246-1489729943941 due to InvalidToken exception. org.apache.hadoop.security.token.SecretManager$InvalidToken: access control error while attempting to set up short-circuit access to /apps/hbase/data/data/default/cyclops-edges/865c0549943a300f09f5dfcd63fbaa67/s/1979e37ca9294d24bd16cd580fb663ab 2018-12-12 18:09:57,326 INFO [sync.2] wal.FSHLog: Slow sync cost: 105 ms, current pipeline: [DatanodeInfoWithStorage[10.84.197.254:50010,DS-159866b7-1b97-469f-9a22-4b03b3dbbe56,DISK], DatanodeInfoWithStorage[10.84.192.255:50010,DS-26463076-ed2d-4883-9013-2870ce87f281,DISK], DatanodeInfoWithStorage[10.84.192.76:50010,DS-b1b894fc-d1e2-4ccf-8cc7-7ccd649cd507,DISK]] 2018-12-12 18:09:57,391 WARN [B.defaultRpcServer.handler=21,queue=0,port=16020] hdfs.BlockReaderFactory: I/O error constructing remote block reader. java.io.IOException: Got error, status message opReadBlock BP-853897652-10.84.192.246-1489729943941:blk_1093248459_19508530 received exception org.apache.hadoop.hdfs.server.datanode.ReplicaNotFoundException: Replica not found for BP-853897652-10.84.192.246-1489729943941:blk_1093248459_19508530, for OP_READ_BLOCK, self=/10.84.197.254:13978, remote=/10.84.192.246:50010, for file /apps/hbase/data/data/default/cyclops-audits-dedup/e44738e4889089bcecf58b66878a7501/l/457245b1ea724f7f80bab245ea6c0604, for pool BP-853897652-10.84.192.246-1489729943941 block 1093248459_19508530 at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:140) at org.apache.hadoop.hdfs.RemoteBlockReader2.checkSuccess(RemoteBlockReader2.java:456) at org.apache.hadoop.hdfs.RemoteBlockReader2.newBlockReader(RemoteBlockReader2.java:424) at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReader(BlockReaderFactory.java:816) at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:695) at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:355) at org.apache.hadoop.hdfs.DFSInputStream.actualGetFromOneDataNode(DFSInputStream.java:1181) at org.apache.hadoop.hdfs.DFSInputStream.fetchBlockByteRange(DFSInputStream.java:1118) at org.apache.hadoop.hdfs.DFSInputStream.pread(DFSInputStream.java:1478) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1441) at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:92) at org.apache.hadoop.hbase.io.hfile.HFileBlock.positionalReadWithExtra(HFileBlock.java:722) at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1420) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1625) at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1504) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:441) at org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:269) at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:642) at java.lang.Thread.run(Thread.java:745) 2018-12-12 18:09:57,391 WARN [B.defaultRpcServer.handler=21,queue=0,port=16020] hdfs.DFSClient: Connection failure: Failed to connect to /10.84.192.246:50010 for file /apps/hbase/data/data/default/cyclops-audits-dedup/e44738e4889089bcecf58b66878a7501/l/457245b1ea724f7f80bab245ea6c0604 for block BP-853897652-10.84.192.246-1489729943941:blk_1093248459_19508530:java.io.IOException: Got error, status message opReadBlock BP-853897652-10.84.192.246-1489729943941:blk_1093248459_19508530 received exception org.apache.hadoop.hdfs.server.datanode.ReplicaNotFoundException: Replica not found for BP-853897652-10.84.192.246-1489729943941:blk_1093248459_19508530, for OP_READ_BLOCK, self=/10.84.197.254:13978, remote=/10.84.192.246:50010, for file /apps/hbase/data/data/default/cyclops-audits-dedup/e44738e4889089bcecf58b66878a7501/l/457245b1ea724f7f80bab245ea6c0604, for pool BP-853897652-10.84.192.246-1489729943941 block 1093248459_19508530 java.io.IOException: Got error, status message opReadBlock BP-853897652-10.84.192.246-1489729943941:blk_1093248459_19508530 received exception org.apache.hadoop.hdfs.server.datanode.ReplicaNotFoundException: Replica not found for BP-853897652-10.84.192.246-1489729943941:blk_1093248459_19508530, for OP_READ_BLOCK, self=/10.84.197.254:13978, remote=/10.84.192.246:50010, for file /apps/hbase/data/data/default/cyclops-audits-dedup/e44738e4889089bcecf58b66878a7501/l/457245b1ea724f7f80bab245ea6c0604, for pool BP-853897652-10.84.192.246-1489729943941 block 1093248459_19508530 at org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:140) The issue that we are facing is very slow Hbase compaction. HDP Version: HDP-2.4.3.0-227 HBase: 1.1.2.2.4 Thanks, Shesh

Online	Offline
Last Visited	‎06-09-2021 02:05 AM

Member Since	‎05-09-2018 08:53 AM
Last Visited	‎06-09-2021 02:05 AM
Posts	44
Kudos received	3

Cloudera Community

Re: How to add LDAP users to Hue via API call

How to add LDAP users to Hue via API call

Re: Why does Standby Namenode read from Journals N...

Re: Unable to lookup the cluster by ID; assuming t...

Why does Standby Namenode read from Journals Nodes...

Unable to lookup the cluster by ID; assuming that ...

Re: hbase 2.0 table stuck in 'DISABLING' state and...

Hbase table is stuck in "Disabling" state. Neither...

Re: How to Configure Ranger and Usersync for LDAP ...

Seeing this error in Hbase Region Server logs: "hd...