Member since
05-09-2016
421
Posts
54
Kudos Received
32
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3747 | 04-22-2022 11:31 AM | |
| 4296 | 01-20-2022 11:24 AM | |
| 3471 | 11-23-2021 12:53 PM |
05-28-2022
11:27 AM
It's seems without space. #HDP DEV Cluster kadmin.local : addprinc krbtgt/HDPDQA.QA.COM@HDPDEV.DEV.COM kadmin.local : addprinc krbtgt/HDPDDEV.DEV.COM@HDPQA.QA.COM #HDP QA cluster Kadmin.local : addprinc krbtgt/HDPDQA.QA.COM@HDPDEV.QA.COM kadmin.local : addprinc krbtgt/HDPDDEV.DEV.COM@HDPQA.QA.COM
... View more
04-25-2022
06:04 AM
That worked! Thanks! Cloudera suggested a patch to Hive found here: https://issues.apache.org/jira/browse/HIVE-23501 But this is much easier! Thanks again!
... View more
04-20-2022
05:13 AM
FYI @rpathak i'm using DbViZ 12.1.3 And hdp 3.1.5
... View more
12-16-2021
11:52 AM
@MikeB Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.
... View more
11-26-2021
01:25 AM
Thanks @rpathak - having discussed this further amongst our team we think we are going to try setting up elastic YARN queues to help this situation.
... View more
05-25-2017
07:52 PM
1 Kudo
Currently ranger does not support defining password of default admin users(rangerusersync and rangertagsync) during installation.
This is a security concern if you miss to update the password post installation as default password for these users is same as username, thereby compromising admin access to ranger.
There is plan to address this in future HDP release by allowing to define the passwords for admin users during installation, however for the time being below workaround can be used to enforce password policy.
Once ranger installation is successful one can use below rest api get and put command to change the password of any user.
This is helpful in case of automated cluster installation using ambari blueprint.
To change the password of the user first get the user details using below call: curl -s -u admin:admin -H "Accept: application/json" -H "Content-Type: application/json" -X GET http://h2.openstacklocal:6080/service/xusers/users/2
You will get something like this. {"id":2,"createDate":"2017-05-24T17:36:49Z","updateDate":"2017-05-25T08:09:25Z","owner":"Admin","updatedBy":"Admin","name":"rangerusersync","firstName":"rangerusersync","lastName":"","password":"*******","description":"rangerusersync","groupIdList":[],"groupNameList":[],"status":1,"isVisible":1,"userSource":0,"userRoleList":["ROLE_SYS_ADMIN"]}
Now send the PUT request with the above information after setting required password as give below: curl -iv -u admin:admin -X PUT -H "Accept: application/json" -H "Content-Type: application/json" http://h2.openstacklocal:6080/service/xusers/secure/users/2 -d '{"id":2,"createDate":"2017-05-24T17:36:49Z","updateDate":"2017-05-25T08:09:25Z","owner":"Admin","updatedBy":"Admin","name":"rangerusersync","firstName":"rangerusersync","lastName":"","password":"Password@123","description":"rangerusersync","groupIdList":[],"groupNameList":[],"status":1,"isVisible":1,"userSource":0,"userRoleList":["ROLE_SYS_ADMIN"]}'
PS : To change password of keyadmin user, use keyadmin credentials in REST api call. Thanks @Ronak bansal for testing this.
... View more
Labels:
03-21-2017
09:46 AM
SYMPTOM
Concurrency issues hit during multi-threaded moveFile issued when processing queries such as "INSERT OVERWRITE TABLE ... SELECT .."
The following pattern is displayed in stack trace: Loading data to table testdb.test_table from hdfs://xyz/ra_hadoop/.hive-staging_hive_2017-01-31_14-09-52_561_8101886747064006778-4/-ext-10000
ERROR : Failed with exception java.util.ConcurrentModificationException
org.apache.hadoop.hive.ql.metadata.HiveException: java.util.ConcurrentModificationException
at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2883)
at org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:3140)
at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1727)
at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:353)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1745)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1491)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1289)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1156)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1151) ROOT CAUSE This issue occurs because of issue described in HIVE-15355.
WORKAROUND
Set the following property at the client side: set hive.mv.files.thread=0;
Re-run the hive query Note: The fix for HIVE-15355 is expected to be included in the next major release of HDP.
... View more
12-28-2016
12:22 PM
1 Kudo
Check auth_to_local rules for Atlas. In many installation there exist invalid rule like below: RULE:[1:$1@$0](.*@EXAMPLE.COM)s/@.*// After removing above rule Atlas UI will work fine.
... View more
Labels:
06-17-2016
12:42 PM
1 Kudo
In order to distcp between two HDFS HA cluster (for example A and B), using nameservice id or to setup falcon clusters having namenode ha, these settings are needed.
Assuming nameservice for cluster A and B is HAA and HAB respectively.
One need to set following properties in hdfs-site.xml
Add value of the nameservices of both clusters in dfs.nameservices. This needs to be done in both the clusters. dfs.nameservices=HAA,HAB Add property dfs.internal.nameservices
In cluster A:
dfs.internal.nameservices = HAA
In cluster B:
dfs.internal.nameservices = HAB
Add dfs.ha.namenodes.<nameservice>. dfs.ha.namenodes.HAB=nn1,nn2 dfs.ha.namenodes.HAA=nn1,nn2 Add property dfs.namenode.rpc-address.<nameservice>.<nn>. dfs.namenode.rpc-address.HAB.nn1 = <NN1_fqdn>:8020 dfs.namenode.rpc-address.HAB.nn2 = <NN2_fqdn>:8020 dfs.namenode.rpc-address.HAA.nn1 = <NN1_fqdn>:8020 dfs.namenode.rpc-address.HAA.nn2 = <NN2_fqdn>:8020
Add property dfs.client.failover.proxy.provider.<nameservice> In cluster A
dfs.client.failover.proxy.provider.HAB = org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider In cluster B
dfs.client.failover.proxy.provider.HAA = org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider Restart HDFS service.
Once complete you will be able to run the distcp command using the nameservice similar to:
hadoop distcp hdfs://HAA/tmp/file1 hdfs://HAB/tmp/
... View more
Labels: